Use of hierarchical models to evaluate performance of cardiac surgery centres in the Italian CABG outcome study

Ten-year outcomes after off-pump and on-pump coronary artery bypass grafting: an inverse probability of treatment weighting comparative study

AIMS

The debate on the advantages and limitations of off-pump myocardial revascularization (OPCAB) on long-term outcomes has not still arrived to a conclusion. This study was designed to compare the impact of OPCAB vs, on-pump coronary artery bypass grafting (CABG) on long-term mortality and major adverse cardiac and cerebrovascular events (MACCEs).

METHODS

The PRIORITY project was designed to evaluate the long-term outcomes of two large prospective multicenter cohort studies on CABG. Data on isolated CABG were linked to two administrative datasets. The inverse probability of treatment weight was employed to balance the treatment groups. Time-to-event methods were employed to analyze endpoints.

RESULTS

The cohort consisted of 10 988 patients who underwent isolated CABG (27.2% OPCAB). The median follow-up time was 7.9 years and was 100% complete. Unadjusted long-term survival was significantly worst for OPCAB, confirmed by weighted models (hazard ratio 1.08, 95% confidence interval (CI) 1.01-1.14, P = 0.01). OPCAB was associated to an increased risk of MACCE at 10 years (weighted hazard ratio 1.18, 95% CI 1.12-1.23, P < 0.001). Inside the MACCEs, OPCAB was significantly related to increased incidence of repeat revascularization (hazard ratio 2.27, 95% CI 1.39-3.85, P < 0.001, in the first 6 months, hazard ratio 1.19, 95% CI 1.09-1.32, P < 0.001 afterward) and stroke (hazard ratio 1.22, 95% CI 1.10-1.35, P < 0.001).

CONCLUSION

The results of this study suggest that OPCAB was associated with an increased risk of mortality, repeat myocardial revascularization and stroke at 10 years compared with on-pump CABG.

Impact of gender on 10-year outcome after coronary artery bypass grafting

 

OBJECTIVES

Our goal was to evaluate the impact of gender on the 10-year outcome of patients after isolated coronary artery bypass grafting (CABG) included in the Italian nationwide PRedictIng long-term Outcomes afteR Isolated coronary arTery bypass surgery (PRIORITY) study.

METHODS

The PRIORITY project was designed to evaluate the long-term outcomes of patients who underwent CABG and were included in 2 prospective multicentre cohort studies. The primary end point of this analysis was major adverse cardiac and cerebrovascular events. Baseline differences between the study groups were balanced with propensity score matching and inverse probability of treatment. Time to events was analysed using Cox regression and competing risk analysis.

RESULTS

The study population comprised 10 989 patients who underwent isolated CABG (women 19.6%). Propensity score matching produced 1898 well-balanced pairs. The hazard of major adverse cardiac and cerebrovascular event was higher in women compared to men [adjusted hazard ratio (HR) 1.13, 95% confidence interval (CI) 1.03-1.23; P = 0.009]. The incidence of major adverse cardiac and cerebrovascular event in women was significantly higher at 1 year (HR 1.31, 95% CI 1.11-1.55; P < 0.001) and after 1 year (HR 1.11, 95% CI 1.00-1.24; P = 0.05). Mortality at 10 years in the matched groups was comparable (HR 1.04, 95% CI 0.93-1.16; P = 0.531). Women have significantly a higher 10-year risk of myocardial infarction (adjusted HR 1.40, 95% CI 1.17-1.68; P = 0.002) and percutaneous coronary intervention (adjusted HR 1.32, 95% CI 1.10-1.59; P = 0.003).

CONCLUSIONS

The present study documented an excess of non-fatal cardiac events after CABG among women despite comparable 10-year survival with men. These findings suggest that studies investigating measures of tertiary prevention are needed to decrease the risk of adverse cardiovascular events among women.

Association between job-related stress and experience of presenteeism among Korean workers stratified on the presence of depression

Background

Presenteeism refers to the phenomenon of working while sick. Its development can be attributed to not only somatic symptoms but also underlying social agreements and workplace atmosphere. In this study, we analyzed presenteeism among workers from various industries, focusing on job-related stress with stratification on the presence of depression.

Methods

We conducted the study with data from questionnaires filled in by different enterprises enrolled in the Federation of Korean Trade Unions. Workers' depressive symptoms were investigated using the Patient Health Questionnaire-2, while questions on job-related stress and presenteeism were derived from the short form of the Korean Occupational Stress Scale and the official Korean version of the Work-Productivity and Activity Impairment Questionnaire-General Health, respectively. Multilevel logistic analysis was conducted to determine the statistical differences derived from the differences between companies.

Results

In total, 930 participants (753 men and 177 women) from 59 enterprises participated in the research. We conducted multilevel logistic regression to determine the association between the variables and presenteeism, with stratification by the presence of depression. Higher job demands and higher interpersonal conflict showed significantly elevated odds ratios (ORs) in univariate models and in the multivariate multilevel model. In the final model of total population, fully adjusted by general and work-related characteristics, higher job demands (OR: 3.29, 95% confidence interval [CI]: 2.08-5.21) and interpersonal conflict (OR: 1.87, 95% CI: 1.29-2.71) had significantly higher ORs-a tendency that remained in participants without depression.

Conclusions

This study reflected the factors associated with presenteeism among workers from various enterprises. The findings revealed that job-related stress was closely related to presenteeism in both the total population and in the population without depression. Thus, it emphasized interventions for managing job stress among workers to reduce presenteeism in general workers' population.

Evaluation of a social protection policy on tuberculosis treatment outcomes: A prospective cohort study

BACKGROUND

Tuberculosis (TB) still represents a major public health problem in Latin America, with low success and high default rates. Poor adherence represents a major threat for TB control and promotes emergence of drug-resistant TB. Expanding social protection programs could have a substantial effect on the global burden of TB; however, there is little evidence to evaluate the outcomes of socioeconomic support interventions. This study evaluated the effect of a conditional cash transfer (CCT) policy on treatment success and default rates in a prospective cohort of socioeconomically disadvantaged patients.

METHODS AND FINDINGS

Data were collected on adult patients with first diagnosis of pulmonary TB starting treatment in public healthcare facilities (HCFs) from 16 health departments with high TB burden in Buenos Aires who were followed until treatment completion or abandonment. The main exposure of interest was the registration to receive the CCT. Other covariates, such as sociodemographic and clinical variables and HCFs' characteristics usually associated with treatment adherence and outcomes, were also considered in the analysis. We used hierarchical models, propensity score (PS) matching, and inverse probability weighting (IPW) to estimate treatment effects, adjusting for individual and health system confounders. Of 941 patients with known CCT status, 377 registered for the program showed significantly higher success rates (82% versus 69%) and lower default rates (11% versus 20%). After controlling for individual and system characteristics and modality of treatment, odds ratio (OR) for success was 2.9 (95% CI 2, 4.3, P < 0.001) and default was 0.36 (95% CI 0.23, 0.57, P < 0.001). As this is an observational study evaluating an intervention not randomly assigned, there might be some unmeasured residual confounding. Although it is possible that a small number of patients was not registered into the program because they were deemed not eligible, the majority of patients fulfilled the requirements and were not registered because of different reasons. Since the information on the CCT was collected at the end of the study, we do not know the exact timing for when each patient was registered for the program.

CONCLUSIONS

The CCT appears to be a valuable health policy intervention to improve TB treatment outcomes. Incorporating these interventions as established policies may have a considerable effect on the control of TB in similar high-burden areas.

Regional variation in healthcare spending and mortality among senior high-cost healthcare users in Ontario, Canada: a retrospective matched cohort study

Background

Senior high cost health care users (HCU) are a priority for many governments. Little research has addressed regional variation of HCU incidence and outcomes, especially among incident HCU. This study describes the regional variation in healthcare costs and mortality across Ontario’s health planning districts [Local Health Integration Networks (LHIN)] among senior incident HCU and non-HCU and explores the relationship between healthcare spending and mortality.

Methods

We conducted a retrospective population-based matched cohort study of incident senior HCU defined as Ontarians aged ≥66 years in the top 5% most costly healthcare users in fiscal year (FY) 2013. We matched HCU to non-HCU (1:3) based on age, sex and LHIN. Primary outcomes were LHIN-based variation in costs (total and 12 cost components) and mortality during FY2013 as measured by variance estimates derived from multi-level models. Outcomes were risk-adjusted for age, sex, ADGs, and low-income status. In a cost-mortality analysis by LHIN, risk-adjusted random effects for total costs and mortality were graphically presented together in a cost-mortality plane to identify low and high performers.

Results

We studied 175,847 incident HCU and 527,541 matched non-HCU. On average, 94 out of 1000 seniors per LHIN were HCU (CV = 4.6%). The mean total costs for HCU in FY2013 were 12 times higher that of non-HCU ($29,779 vs. $2472 respectively), whereas all-cause mortality was 13.6 times greater (103.9 vs. 7.5 per 1000 seniors).

Regional variation in costs and mortality was lower in senior HCU compared with non-HCU. We identified greater variability in accessing the healthcare system, but, once the patient entered the system, variation in costs was low. The traditional drivers of costs and mortality that we adjusted for played little role in driving the observed variation in HCUs’ outcomes. We identified LHINs that had high mortality rates despite elevated healthcare expenditures and those that achieved lower mortality at lower costs. Some LHINs achieved low mortality at excessively high costs.

Conclusions

Risk-adjusted allocation of healthcare resources to seniors in Ontario is overall similar across health districts, more so for HCU than non-HCU. Identified important variation in the cost-mortality relationship across LHINs needs to be further explored.

Electronic supplementary material

The online version of this article (10.1186/s12877-018-0952-7) contains supplementary material, which is available to authorized users.

Enhance quality care performance: Determination of the variables for establishing a common database in French paediatric critical care units

Selected variables for the French Paediatric Intensive Care registry.

RATIONALE, AIMS, AND OBJECTIVES

Providing quality care requires follow-up in regard to clinical and economic activities. Over the past decade, medical databases and patient registries have expanded considerably, particularly in paediatric critical care medicine (eg, the Paediatric Intensive Care Audit Network (PICANet) in the UK, the Australian and New Zealand Paediatric Intensive Care (ANZPIC) Registry in Australia and New Zealand, and the Virtual Paediatric Intensive Care Unit Performance System (VPS) in the USA). Such a registry is not yet available in France. The aim of this study was to determine variables that ought to be included in a French paediatric critical care registry.

METHODS

Variables, items, and subitems from 3 foreign registries and 2 French local databases were used. Items described each variable, and subitems described items. The Delphi method was used to evaluate and rate 65 variables, 90 items, and 17 subitems taking into account importance or relevance based on input from 28 French physicians affiliated with the French Paediatric Critical Care Group. Two ratings were used between January and May 2013.

RESULTS

Fifteen files from 10 paediatric intensive care units were included. Out of 65 potential variables, 48 (74%) were considered to be indispensable, 16 (25%) were considered to be optional, and 1 (2%) was considered to be irrelevant. Out of 90 potential items, 62 (69%) were considered to be relevant, 23 (26%) were considered to be of little relevance, and 5 (6%) were considered to be irrelevant. Out of 17 potential subitems, 9 (53%) were considered to be relevant, 6 (35%) were considered to be of little relevance, and 2 (12%) were considered to be irrelevant.

CONCLUSIONS

The necessary variables that ought to be included in a French paediatric critical care registry were identified. The challenge now is to develop the French registry for paediatric intensive care units.

Early mortality following percutaneous coronary intervention and cardiac surgery: Correlations within providers and operators

BACKGROUND

It is not clear whether correlations exist within hospitals or operators among risk-adjusted mortality rates (RAMRs) for the most common cardiac interventions and how much of variations in outcomes are residually explained by providers and physicians. We examined these aspects by using recent national data on percutaneous coronary intervention (PCI) and cardiac surgery.

METHODS

Publically available data from New York State aggregated at hospital and operator level were downloaded by Department of Health website for in-hospital/30-day mortality after PCI, coronary artery bypass graft (CABG) and valve surgery. Correlations between RAMRs were evaluated by using Spearman's coefficient (rho). The proportion of mortality variation attributed to hospitals and operators was estimated.

RESULTS

During the period 2008-2013, 390 cardiologists from 63 hospitals and 163 surgeons from 41 centres were evaluated. The RAMRs during 2008-2010 correlated with the RAMRs during 2011-2013 for valve surgery within providers (rho=0.55;p<0.001) and within interventionists for PCI (rho=0.21;p<0.001), isolated CABG (rho=0.25;p=0.009), and any valve surgery or CABG procedure (rho=0.49;p<0.001). The most recent hospital's RAMRs (year 2012 and 2013) significantly correlated in PCI (rho=0.40;p=0.002) but not in CABG (rho=0.13;p=0.413). <2% of mortality variations was attributed to providers and 2-3% to difference between operators.

CONCLUSIONS

A correlation exists at provider and operator level in RAMRs for PCI and cardiac surgery procedures performed in New York State. Beyond patient's risk profile, that is the strongest predictor of early mortality after a cardiac procedure, hospitals and operators have a small but statistically significant contribution to variation in post-operative outcome.

Rethinking Thirty-Day Hospital Readmissions: Shorter Intervals Might Be Better Indicators Of Quality Of Care

Public reporting and payment programs in the United States have embraced thirty-day readmissions as an indicator of between-hospital variation in the quality of care, despite limited evidence supporting this interval. We examined risk-standardized thirty-day risk of unplanned inpatient readmission at the hospital level for Medicare patients ages sixty-five and older in four states and for three conditions: acute myocardial infarction, heart failure, and pneumonia. The hospital-level quality signal captured in readmission risk was highest on the first day after discharge and declined rapidly until it reached a nadir at seven days, as indicated by a decreasing intracluster correlation coefficient. Similar patterns were seen across states and diagnoses. The rapid decay in the quality signal suggests that most readmissions after the seventh day postdischarge were explained by community- and household-level factors beyond hospitals' control. Shorter intervals of seven or fewer days might improve the accuracy and equity of readmissions as a measure of hospital quality for public accountability.

Risk Adjustment for Lumbar Dysfunction: Comparison of Linear Mixed Models With and Without Inclusion of Between-Clinic Variation as a Random Effect

BACKGROUND

Valid comparison of patient outcomes of physical therapy care requires risk adjustment for patient characteristics using statistical models. Because patients are clustered within clinics, results of risk adjustment models are likely to be biased by random, unobserved between-clinic differences. Such bias could lead to inaccurate prediction and interpretation of outcomes.

PURPOSE

The purpose of this study was to determine if including between-clinic variation as a random effect would improve the performance of a risk adjustment model for patient outcomes following physical therapy for low back dysfunction.

DESIGN

This was a secondary analysis of data from a longitudinal cohort of 147,623 patients with lumbar dysfunction receiving physical therapy in 1,470 clinics in 48 states of the United States.

METHODS

Three linear mixed models predicting patients' functional status (FS) at discharge, controlling for FS at intake, age, sex, number of comorbidities, surgical history, and health care payer, were developed. Models were: (1) a fixed-effect model, (2) a random-intercept model that allowed clinics to have different intercepts, and (3) a random-slope model that allowed different intercepts and slopes for each clinic. Goodness of fit, residual error, and coefficient estimates were compared across the models.

RESULTS

The random-effect model fit the data better and explained an additional 11% to 12% of the between-patient differences compared with the fixed-effect model. Effects of payer, acuity, and number of comorbidities were confounded by random clinic effects.

LIMITATIONS

Models may not have included some variables associated with FS at discharge. The clinics studied may not be representative of all US physical therapy clinics.

CONCLUSIONS

Risk adjustment models for functional outcome of patients with lumbar dysfunction that control for between-clinic variation performed better than a model that does not.

Accuracy of EuroSCORE II in patients undergoing minimally invasive mitral valve surgery

OBJECTIVES

EuroSCORE II has been implemented with the view to providing better performance than the previous logistic EuroSCORE. However, until now, no external validations have been carried out in the minimally invasive context. Therefore, we sought to validate the accuracy of EuroSCORE II in a retrospective series of consecutive patients undergoing minimally invasive mitral valve surgery.

METHODS

Data of 1609 consecutive patients who underwent minimally invasive mitral valve surgery in our institution were retrospectively reviewed. The accuracy of EuroSCORE II was assessed in terms of discrimination and calibration. Discrimination was tested via analysis of the area under the curve of receiver operator characteristic; calibration was achieved by calculating the observed versus expected mortality ratio and the Hosmer-Lemeshow test for test probability; global accuracy was assessed by using Brier's score; results were compared with the previous logistic EuroSCORE version. EuroSCORE II performance was also tested for discrimination of postoperative complications. Discrimination subgroup analysis was carried out for single surgeon results, and for high-risk patients those outliers were defined after boxplot analysis (EuroSCORE II ≥6%).

RESULTS

EuroSCORE II showed good discrimination power (area under the curve 0.846), and was statistically superior to logistic EuroSCORE (P = 0.01). In terms of calibration, both EuroSCORE II and logistic over-predicted mortality; with regard to adverse events, the discrimination of EuroSCORE II was adequate for acute renal failure, low-output syndrome and increased intensive care unit stay; area under the curve of receiver operating characteristic for high-risk patients with EuroSCORE ≥6% was suboptimal (0.654); single surgeon results did not influence the discrimination of EuroSCORE II.

CONCLUSIONS

EuroSCORE II showed good discrimination power in our series of minimally invasive mitral valve patients; however, it over-predicted mortality. Individual performance did not influence discrimination. Performance was suboptimal for prediction of complications and for high-risk subgroup in-hospital mortality.

The application of a novel 'rising activity, multi-level mixed effects, indicator emphasis' (RAMMIE) method for syndromic surveillance in England

MOTIVATION

Syndromic surveillance is the real-time collection and interpretation of data to allow the early identification of public health threats and their impact, enabling public health action. The 'rising activity, multi-level mixed effects, indicator emphasis' method was developed to provide a single robust method enabling detection of unusual activity across a wide range of syndromes, nationally and locally.

RESULTS

The method is shown here to have a high sensitivity (92%) and specificity (99%) compared to previous methods, whilst halving the time taken to detect increased activity to 1.3 days.

AVAILABILITY AND IMPLEMENTATION

The method has been applied successfully to syndromic surveillance systems in England providing realistic models for baseline activity and utilizing prioritization rules to ensure a manageable number of 'alarms' each day.

CONTACT

roger.morbey@phe.gov.uk.

Classifying hospitals as mortality outliers: logistic versus hierarchical logistic models

The use of hierarchical logistic regression for provider profiling has been recommended due to the clustering of patients within hospitals, but has some associated difficulties. We assess changes in hospital outlier status based on standard logistic versus hierarchical logistic modelling of mortality. The study population consisted of all patients admitted to acute, non-specialist hospitals in England between 2007 and 2011 with a primary diagnosis of acute myocardial infarction, acute cerebrovascular disease or fracture of neck of femur or a primary procedure of coronary artery bypass graft or repair of abdominal aortic aneurysm. We compared standardised mortality ratios (SMRs) from non-hierarchical models with SMRs from hierarchical models, without and with shrinkage estimates of the predicted probabilities (Model 1 and Model 2). The SMRs from standard logistic and hierarchical models were highly statistically significantly correlated (r > 0.91, p = 0.01). More outliers were recorded in the standard logistic regression than hierarchical modelling only when using shrinkage estimates (Model 2): 21 hospitals (out of a cumulative number of 565 pairs of hospitals under study) changed from a low outlier and 8 hospitals changed from a high outlier based on the logistic regression to a not-an-outlier based on shrinkage estimates. Both standard logistic and hierarchical modelling have identified nearly the same hospitals as mortality outliers. The choice of methodological approach should, however, also consider whether the modelling aim is judgment or improvement, as shrinkage may be more appropriate for the former than the latter.

Risk stratification for in-hospital mortality after cardiac surgery: external validation of EuroSCORE II in a prospective regional registry

OBJECTIVES

To evaluate performance of the European System for Cardiac Operation Risk Evaluation (EuroSCORE II), to assess the influence of model updating and to derive a hierarchical tree for modelling the relationship between EuroSCORE II risk factors and hospital mortality after cardiac surgery in a large prospective contemporary cohort of consecutive adult patients.

METHODS

Data on consecutive patients, who underwent on-pump cardiac surgery or off-pump coronary artery bypass graft intervention, were retrieved from Puglia Adult Cardiac Surgery Registry. Discrimination, calibration, re-estimation of EuroSCORE II coefficients and hierarchical tree analysis of risk factors were assessed.

RESULTS

Out 6293 procedures, 6191 (98.4%) had complete data for EuroSCORE II assessment with a hospital mortality rate of 4.85% and EuroSCORE II of 4.40 ± 7.04%. The area under the receiver operator characteristic curve (0.830) showed good discriminative ability of EuroSCORE II in distinguishing patients who died and those who survived. Calibration of EuroSCORE II was preserved with lower predicted than observed risk in the highest EuroSCORE II deciles. At logistic regression analysis, the complete revision of the model had most of re-estimated regression coefficients not statistically different from those in the original EuroSCORE II model. When missing values were replaced with the mean EuroSCORE II value according to urgency and weight of intervention, the risk score confirmed discrimination and calibration obtained over the entire sample. A recursive tree-building algorithm of EuroSCORE II variables identified three large groups (55.1, 17.1 and 18.1% of procedures) with low-to-moderate risk (observed mortality of 1.5, 3.2 and 6.4%) and two groups (3.8 and 5.9% of procedures) at high risk (mortality of 14.6 and 32.2%). Patients with low-to-moderate risk had good agreement between observed events and predicted frequencies by EuroSCORE II, whereas those at greater risk showed an underestimation of expected mortality.

CONCLUSIONS

This study demonstrates that EuroSCORE II is a good predictor of hospital mortality after cardiac surgery in an external validation cohort of contemporary patients from a multicentre prospective regional registry. The EuroSCORE II predicts hospital mortality with a slight underestimation in high-risk patients that should be further and better evaluated. The EuroSCORE II variables as a risk tree provides clinicians and surgeons a practical bedside tool for mortality risk stratification of patients at low, intermediate and high risk for hospital mortality after cardiac surgery.

Assessing the accuracy of profiling methods for identifying top providers: performance of mental health care providers

Provider profiling as a means to describe and compare the performance of health care professionals has gained momentum in the past decade. As a key component of pay-for-performance programs profiling has been increasingly used to identify top-performing providers. However, rigorous examination of the performance of statistical methods for profiling when used to classify top-performing providers is lacking. The objective of this study was to compare the classification accuracy of three methods for identifying providers exceeding performance thresholds and to analyze data on satisfaction with mental health care providers at Group Health Cooperative using these methods. Questionnaire data on patient satisfaction with mental health care providers at Group Health Cooperative was collected between April 2008 and January 2010. Simulated data were used to compare the classification accuracy of alternative statistical methods. We evaluated sensitivity, specificity, and root mean squared error of alternative statistical methods using simulated data. For Group Health providers, we compared agreement of alternative approaches to classification. We found that when between-provider variability in performance was low, all three methods exhibited poor classification accuracy. When used to evaluate mental health care provider performance, we found substantial uncertainty in the estimates and poor agreement across methods. Based on these findings, we recommend providing uncertainty estimates for provider rankings and caution against the use of any classification method when between-provider variability is low.

Hospital-level associations with 30-day patient mortality after cardiac surgery: a tutorial on the application and interpretation of marginal and multilevel logistic regression

Background

Marginal and multilevel logistic regression methods can estimate associations between hospital-level factors and patient-level 30-day mortality outcomes after cardiac surgery. However, it is not widely understood how the interpretation of hospital-level effects differs between these methods.

Methods

The Australasian Society of Cardiac and Thoracic Surgeons (ASCTS) registry provided data on 32,354 patients undergoing cardiac surgery in 18 hospitals from 2001 to 2009. The logistic regression methods related 30-day mortality after surgery to hospital characteristics with concurrent adjustment for patient characteristics.

Results

Hospital-level mortality rates varied from 1.0% to 4.1% of patients. Ordinary, marginal and multilevel regression methods differed with regard to point estimates and conclusions on statistical significance for hospital-level risk factors; ordinary logistic regression giving inappropriately narrow confidence intervals. The median odds ratio, MOR, from the multilevel model was 1.2 whereas ORs for most patient-level characteristics were of greater magnitude suggesting that unexplained between-hospital variation was not as relevant as patient-level characteristics for understanding mortality rates. For hospital-level characteristics in the multilevel model, 80% interval ORs, IOR-80%, supplemented the usual ORs from the logistic regression. The IOR-80% was (0.8 to 1.8) for academic affiliation and (0.6 to 1.3) for the median annual number of cardiac surgery procedures. The width of these intervals reflected the unexplained variation between hospitals in mortality rates; the inclusion of one in each interval suggested an inability to add meaningfully to explaining variation in mortality rates.

Conclusions

Marginal and multilevel models take different approaches to account for correlation between patients within hospitals and they lead to different interpretations for hospital-level odds ratios.

Logistic versus hierarchical modeling: an analysis of a statewide inpatient sample

BACKGROUND

Although logistic regression is traditionally used to calculate hospital standardized mortality ratio (HSMR), it ignores the hierarchical structure of the data that can exist within a given database. Hierarchical models allow examination of the effect of data clustering on outcomes.

STUDY DESIGN

Traditional logistic regression and random intercepts fixed slopes hierarchical models were fitted to a dataset of patients hospitalized between 2005 and 2007 in Massachusetts. We compared the observed to expected (O/E) in-hospital death ratios between the 2 modeling techniques, a restricted HSMR using only those diagnosis models that converged in both methods and a full hybrid HSMR using a combination of the hierarchical diagnosis models when they converge, plus the remaining diagnoses using standard logistic regression models.

RESULTS

We restricted the analysis to the 36 diagnoses accounting for 80% of in-hospital deaths nationally, based on 1,043,813 admissions (59 hospitals). A failure of the hierarchical models to converge in 15 of 36 diagnosis groups hindered full HSMR comparisons. A restricted HSMR, derived from a dataset based on the 21 diagnosis groups that converged (552,933 admissions) showed very high correlation (Pearson r = 0.99). Both traditional logistic regression and hierarchical model identified 12 statistical outliers in common, 7 with high O/E values and 5 with low O/E values. In addition, the multilevel analysis identified 5 additional unique high outliers and 1 additional unique low outlier, and the conventional model identified 2 additional unique low outliers.

CONCLUSIONS

Similar results were obtained from the 2 modeling techniques in terms of O/E ratios. However, because a hierarchical model is associated with convergence problems, traditional logistic regression remains our recommended procedure for computing HSMRs.

The unintended consequence of diabetes mellitus pay-for-performance (P4P) program in Taiwan: are patients with more comorbidities or more severe conditions likely to be excluded from the P4P program?

OBJECTIVE

Taiwan has instituted a pay-for-performance (P4P) program for diabetes mellitus (DM) patients that rewards doctors based in part on outcomes for their DM patients. Doctors are permitted to choose which of their DM patients are included in the P4P program. We test whether seriously ill DM patients are disproportionately excluded from the P4P program.

DATA SOURCE/STUDY SETTING

This study utilizes data from the National Health Insurance (NHI) database in Taiwan for the period of January 2007 to December 2007. Our sample includes 146,481 DM-P4P patients (16.56 percent of the total) and 737,971 non-DM-P4P patients.

DATA COLLECTION/EXTRACTION METHODS

We use logistic and multilevel models to estimate the effects of patient and hospital characteristics on P4P selection.

PRINCIPAL FINDINGS

The results show that older patients and patients with more comorbidities or more severe conditions are prone to be excluded from P4P programs.

CONCLUSIONS

We found that DM patients are disproportionately excluded from P4P programs. Our results point to the importance of mandated participation and risk adjustment measures in P4P programs.