Statistical Methods for Profiling Providers of Medical Care: Issues and Applications

Risk-standardizing hospital rates of survival for pediatric in-hospital CPR events

BACKGROUND

Bradycardia with poor perfusion is the most common reason for in-hospital cardiopulmonary resuscitation (CPR) in children. To date, validated methods to risk-standardize pediatric survival rates for CPR events in hospitals that includes bradycardia with poor perfusion do not exist.

METHODS

Within Get With the Guidelines-Resuscitation, we identified 8080 children who underwent CPR between 2016 and 2023. Using hierarchical logistic regression, we derived and validated a model for survival to hospital discharge to calculate risk-standardized survival rates (RSSRs) for hospitals.

RESULTS

Bradycardia with poor perfusion comprised 56.4% of pediatric CPR events. An initial full model in the derivation cohort identified 16 predictors of survival (c-statistic = 0.772), and a parsimonious model with 13 predictors maintained good discrimination (c-statistic = 0.769). The model calibrated well in the validation cohort (R2 = 0.993). Final predictors included: age group, illness category, initial rhythm, arrest location, renal insufficiency, hepatic insufficiency, sepsis, metabolic or electrolyte abnormality, metastatic or hematologic malignancy, cardiac acyanotic congenital abnormality, congenital noncardiac abnormality, and mechanical ventilation or continuous intravenous vasopressor at the time of the CPR event. Among 100 hospitals with ≥5 CPR events, the median RSSR was 51.8% (IQR: 50.1-54.5%; range: 40.5-62.9%). The adjusted median OR was 1.32 (95% CI: 1.18-1.43), which suggests that the odds of survival to discharge for two children with similar characteristics varied by 32% between hospitals.

CONCLUSION

We developed and validated a model to risk-standardize hospital rates of survival for children undergoing CPR, including those with bradycardia and poor perfusion. This model can facilitate efforts to benchmark hospitals in resuscitation outcomes for children.

Multilevel Quality Indicators: Methodology and Monte Carlo Evidence

BACKGROUND

Quality indicators are frequently used to assess the performance of health care providers, in particular hospitals. Established approaches to the design of such indicators are subject to distortions due to indirect standardization and high variance of estimators. Indicators for geographical regions are rarely considered.

OBJECTIVES

To develop and evaluate a methodology of multilevel quality indicators (MQIs) for both health care providers and geographical regions.

RESEARCH DESIGN

We formally derived MQIs from a statistical multilevel model, which may include characteristics of patients, providers, and regions. We used Monte Carlo simulation to assess the performance of MQIs relative to established approaches based on the standardized mortality/morbidity ratio (SMR) and the risk-standardized mortality rate (RSMR).

MEASURES

Rank correlation between true provider/region effects and quality indicator estimates; shares of the 10% best and 10% worst providers identified by the quality indicators.

RESULTS

The proposed MQIs are: (1) standardized hospital outcome rate (SHOR); (2) regional SHOR; and (3) regional standardized patient outcome rate. Monte Carlo simulations indicated that the SHOR provides substantially better estimates of provider performance than the SMR and risk-standardized mortality rate in almost all scenarios. The regional standardized patient outcome rate was slightly more stable than the regional SMR. We also found that modeling of regional characteristics generally improves the adequacy of provider-level estimates.

CONCLUSIONS

MQIs methodology facilitates adequate and efficient estimation of quality indicators for both health care providers and geographical regions.

Do Common Risk Adjustment Methods Do Their Job Well If Center Effects Are Correlated With the Center-Specific Mean Values of Patient Characteristics?

BACKGROUND

Direct and indirect standardization are well-established approaches to performing risk adjustment when comparing outcomes between healthcare providers. However, it is an open question whether they work well when there is an association between the center effects and the distributions of the patient characteristics in these centers.

OBJECTIVES AND METHODS

We try to shed further light on the impact of such an association. We construct an artificial case study with a single covariate, in which centers can be classified as performing above, on, or below average, and the center effects correlate with center-specific mean values of a patient characteristic, as a consequence of differential quality improvement. Based on this case study, direct standardization and indirect standardization-based on marginal as well as conditional models-are compared with respect to systematic differences between their results.

RESULTS

Systematic differences between the methods were observed. All methods produced results that partially reflect differences in mean age across the centers. This may mask the classification as above, on, or below average. The differences could be explained by an inspection of the parameter estimates in the models fitted.

CONCLUSIONS

In case of correlations of center effects with center-specific mean values of a covariate, different risk adjustment methods can produce systematically differing results. This suggests the routine use of sensitivity analyses. Center effects in a conditional model need not reflect the position of a center above or below average, questioning its use in defining the truth. Further empirical investigations are necessary to judge the practical relevance of these findings.

Change in Hospital Risk-Standardized Stroke Mortality Performance With and Without the Passive Surveillance Stroke Severity Score

BACKGROUND

Adjustment for baseline stroke severity is necessary for accurate assessment of hospital performance. We evaluated whether adjusting for the Passive Surveillance Stroke SeVerity (PaSSV) score, a measure of stroke severity derived using administrative data, changed hospital-specific estimated 30-day risk-standardized mortality rate (RSMR) after stroke.

METHODS

We used linked administrative data to identify adults who were hospitalized with ischemic stroke or intracerebral hemorrhage across 157 hospitals in Ontario, Canada between 2014 and 2019. We fitted a random effects logistic regression model using Markov Chain Monte Carlo methods to estimate hospital-specific 30-day RSMR and 95% credible intervals with adjustment for age, sex, Charlson comorbidity index, and stroke type. In a separate model, we additionally adjusted for stroke severity using PaSSV. Hospitals were defined as low-performing, average-performing, or high-performing depending on whether the RSMR and 95% credible interval were above, overlapping, or below the cohort's crude mortality rate.

RESULTS

We identified 65,082 patients [48.0% were female, the median age (25th,75th percentiles) was 76 years (65,84), and 86.4% had an ischemic stroke]. The crude 30-day all-cause mortality rate was 14.1%. The inclusion of PaSSV in the model reclassified 18.5% (n=29) of the hospitals. Of the 143 hospitals initially classified as average-performing, after adjustment for PaSSV, 20 were reclassified as high-performing and 8 were reclassified as low-performing. Of the 4 hospitals initially classified as low-performing, 1 was reclassified as high-performing. All 10 hospitals initially classified as high-performing remained unchanged.

CONCLUSION

PaSSV may be useful for risk-adjusting mortality when comparing hospital performance. External validation of our findings in other jurisdictions is needed.

Flagging unusual clusters based on linear mixed models using weighted and self-calibrated predictors

Statistical models incorporating cluster-specific intercepts are commonly used in hierarchical settings, for example, observations clustered within patients or patients clustered within hospitals. Predicted values of these intercepts are often used to identify or "flag" extreme or outlying clusters, such as poorly performing hospitals or patients with rapid declines in their health. We consider a variety of flagging rules, assessing different predictors, and using different accuracy measures. Using theoretical calculations and comprehensive numerical evaluation, we show that previously proposed rules based on the 2 most commonly used predictors, the usual best linear unbiased predictor and fixed effects predictor, perform extremely poorly: the incorrect flagging rates are either unacceptably high (approaching 0.5 in the limit) or overly conservative (eg, much <0.05 for reasonable parameter values, leading to very low correct flagging rates). We develop novel methods for flagging extreme clusters that can control the incorrect flagging rates, including very simple-to-use versions that we call "self-calibrated." The new methods have substantially higher correct flagging rates than previously proposed methods for flagging extreme values, while controlling the incorrect flagging rates. We illustrate their application using data on length of stay in pediatric hospitals for children admitted for asthma diagnoses.

[Hospital admissions and outcomes for systolic and diastolic heart failure in Spain between 2016 and 2019: A population-based study]

BACKGROUND AND PURPOSE

In Spain there is a lack of population data that specifically compare hospitalization for systolic and diastolic heart failure (HF). We assessed clinical characteristics, in-hospital mortality and 30-day cardiovascular readmission rates differentiating by HF type.

METHODS

We conducted a retrospective observational study of patients discharged with the principal diagnosis of HF from The National Health System' acute hospital during 2016-2019, distinguishing between systolic and diastolic HF. The source of the data was the Minimum Basic Data Set. The risk-standardized in-hospital mortality ratio and risk-standardized 30-day cardiovascular readmission ratio were calculated using multilevel risk adjustment models.

RESULTS

The 190,200 episodes of HF were selected. Of these, 163,727 (86.1%) were classified as diastolic HF and were characterized by older age, higher proportion of women, diabetes mellitus, dementia and renal failure than those with systolic HF. In the multilevel risk adjustment models, diastolic HF was a protective factor for both in-hospital mortality (odds ratio [OR]: 0.79; 95% confidence interval [CI]: 0.75-0.83; P<.001) and 30-day cardiovascular readmission versus systolic HF (OR: 0.93; 95% CI: 0.88-0.97; P=.002).

CONCLUSIONS

In Spain, between 2016 and 2019, hospitalization episodes for HF were mostly due to diastolic HF. According to the multilevel risk adjustment models, diastolic HF compared to systolic HF was a protective factor for both in-hospital mortality and 30-day cardiovascular readmission.

Clinical profile, management and outcomes of patients with cardiogenic shock undergoing transfer between centers in Spain

INTRODUCTION AND OBJECTIVES

The aim of this study was to analyze the clinical profile, management, and prognosis of ST segment elevation myocardial infarction-related cardiogenic shock (STEMI-CS) requiring interhospital transfer, as well as the prognostic impact of structural variables of the treating centers in this setting.

METHODS

This study included patients with STEMI-CS treated at revascularization-capable centers from 2016 to 2020. The patients were divided into the following groups: group A: patients attended throughout their admission at hospitals with interventional cardiology without cardiac surgery; group B: patients treated at hospitals with interventional cardiology and cardiac surgery; and group C: patients transferred to centers with interventional cardiology and cardiac surgery. We analyzed the association between the volume of STEMI-CS cases treated, the availability of cardiac intensive care units (CICU), and heart transplant with hospital mortality.

RESULTS

A total of 4189 episodes were included: 1389 (33.2%) from group A, 2627 from group B (62.7%), and 173 from group C (4.1%). Transferred patients were younger, had a higher cardiovascular risk, and more commonly underwent revascularization, mechanical circulatory support, and heart transplant during hospitalization (P<.001). The crude mortality rate was lower in transferred patients (46.2% vs 60.3% in group A and 54.4% in group B, (P<.001)). Lower mortality was associated with a higher volume of care and CICU availability (OR, 0.75, P=.009; and 0.80, P=.047).

CONCLUSIONS

The proportion of transfers in patients with STEMI-CS in our setting is low. Transferred patients were younger and underwent more invasive procedures. Mortality was lower among patients transferred to centers with a higher volume of STEMI-CS cases and CICU.

Competing Risk Modeling with Bivariate Varying Coefficients to Understand the Dynamic Impact of COVID-19

The coronavirus disease 2019 (COVID-19) pandemic has exerted a profound impact on patients with end-stage renal disease relying on kidney dialysis to sustain their lives. A preliminary analysis of dialysis patient postdischarge hospital readmissions and deaths in 2020 revealed that the COVID-19 effect has varied significantly with postdischarge time and time since the pandemic onset. However, the complex dynamics cannot be characterized by existing varying coefficient models. To address this issue, we propose a bivariate varying coefficient model for competing risks, where tensor-product B-splines are used to estimate the surface of the COVID-19 effect. An efficient proximal Newton algorithm is developed to facilitate the fitting of the new model to the massive data for Medicare beneficiaries on dialysis. Difference-based anisotropic penalization is introduced to mitigate model overfitting and effect wiggliness; a cross-validation method is derived to determine optimal tuning parameters. Hypothesis testing procedures are designed to examine whether the COVID-19 effect varies significantly with postdischarge time and the time since the pandemic onset, either jointly or separately. Applications to Medicare dialysis patients demonstrate the real-world performance of the proposed methods. Simulation experiments are conducted to evaluate the estimation accuracy, type I error rate, statistical power, and model selection procedures. Supplementary materials for this article are available online.

Misdiagnosis-related harm quantification through mixture models and harm measures

Investigating and monitoring misdiagnosis-related harm is crucial for improving health care. However, this effort has traditionally focused on the chart review process, which is labor intensive, potentially unstable, and does not scale well. To monitor medical institutes' diagnostic performance and identify areas for improvement in a timely fashion, researchers proposed to leverage the relationship between symptoms and diseases based on electronic health records or claim data. Specifically, the elevated disease risk following a false-negative diagnosis can be used to signal potential harm. However, off-the-shelf statistical methods do not fully accommodate the data structure of a well-hypothesized risk pattern and thus fail to address the unique challenges adequately. To fill these gaps, we proposed a mixture regression model and its associated goodness-of-fit testing. We further proposed harm measures and profiling analysis procedures to quantify, evaluate, and compare misdiagnosis-related harm across institutes with potentially different patient population compositions. We studied the performance of the proposed methods through simulation studies. We then illustrated the methods through data analyses on stroke occurrence data from the Taiwan Longitudinal Health Insurance Database. From the analyses, we quantitatively evaluated risk factors for being harmed due to misdiagnosis, which unveiled some insights for health care quality research. We also compared general and special care hospitals in Taiwan and observed better diagnostic performance in special care hospitals using various new evaluation measures.

Hospital profiling using Bayesian decision theory

For evaluating the quality of care provided by hospitals, special interest lies in the identification of performance outliers. The classification of healthcare providers as outliers or non-outliers is a decision under uncertainty, because the true quality is unknown and can only be inferred from an observed result of a quality indicator. We propose to embed the classification of healthcare providers into a Bayesian decision theoretical framework that enables the derivation of optimal decision rules with respect to the expected decision consequences. We propose paradigmatic utility functions for two typical purposes of hospital profiling: the external reporting of healthcare quality and the initiation of change in care delivery. We make use of funnel plots to illustrate and compare the resulting optimal decision rules and argue that sensitivity and specificity of the resulting decision rules should be analyzed. We then apply the proposed methodology to the area of hip replacement surgeries by analyzing data from 1,277 hospitals in Germany which performed over 180,000 such procedures in 2017. Our setting illustrates that the classification of outliers can be highly dependent upon the underlying utilities. We conclude that analyzing the classification of hospitals as a decision theoretic problem helps to derive transparent and justifiable decision rules. The methodology for classifying quality indicator results is implemented in an R package (iqtigbdt) and is available on GitHub.

Using the Kitagawa Decomposition to Measure Overall-and Individual Facility Contributions to-Within-facility and Between-facility Differences: Analyzing Racial and Ethnic Wait Time Disparities in the Veterans Health Administration

BACKGROUND

Identifying whether differences in health care disparities are due to within-facility or between-facility differences is key to disparity reductions. The Kitagawa decomposition divides the difference between 2 means into within-facility differences and between-facility differences that are measured on the same scale as the original disparity. It also enables the identification of facilities that contribute most to within-facility differences (based on facility-level disparities and the proportion of patient population served) and between-facility differences.

OBJECTIVES

Illustrate the value of a 2-stage Kitagawa decomposition to partition a disparity into within-facility and between-facility differences and to measure the contribution of individual facilities to each type of difference.

SUBJECTS

Veterans receiving a new outpatient consult for cardiology or orthopedic services during fiscal years 2019-2021.

MEASURES

Wait time for a new-patient consult.

METHODS

In stage 1, we predicted wait time for each Veteran from a multivariable model; in stage 2, we aggregated individual predictions to determine mean adjusted wait times for Hispanic, Black, and White Veterans and then decomposed differences in wait times between White Veterans and each of the other groups.

RESULTS

Noticeably longer wait times were experienced by Hispanic Veterans for cardiology (2.32 d, 6.8% longer) and Black Veterans for orthopedics (3.49 d, 10.3% longer) in both cases due entirely to within-facility differences. The results for Hispanic Veterans using orthopedics illustrate how positive within-facility differences (0.57 d) can be offset by negative between-facility differences (-0.34 d), resulting in a smaller overall disparity (0.23 d). Selecting 10 facilities for interventions in orthopedics based on the largest contributions to within-in facility differences instead of the largest disparities resulted in a higher percentage of Veterans impacted (31% and 12% of Black and White Veterans, respectively, versus 9% and 10% of Black and White Veterans, respectively) and explained 21% of the overall within-facility difference versus 11%.

CONCLUSIONS

The Kitagawa approach allows the identification of disparities that might otherwise be undetected. It also allows the targeting of interventions at those facilities where improvements will have the largest impact on the overall disparity.

The impact of surgical volume on hospital ranking using the standardized infection ratio

The Centers for Medicare and Medicaid Services require hospitals to report on quality metrics which are used to financially penalize those that perform in the lowest quartile. Surgical site infections (SSIs) are a critical component of the quality metrics that target healthcare-associated infections. However, the accuracy of such hospital profiling is highly affected by small surgical volumes which lead to a large amount of uncertainty in estimating standardized hospital-specific infection rates. Currently, hospitals with less than one expected SSI are excluded from rankings, but the effectiveness of this exclusion criterion is unknown. Tools that can quantify the classification accuracy and can determine the minimal surgical volume required for a desired level of accuracy are lacking. We investigate the effect of surgical volume on the accuracy of identifying poorly performing hospitals based on the standardized infection ratio and develop simulation-based algorithms for quantifying the classification accuracy. We apply our proposed method to data from HCA Healthcare (2014-2016) on SSIs in colon surgery patients. We estimate that for a procedure like colon surgery with an overall SSI rate of 3%, to rank hospitals in the HCA colon SSI dataset, hospitals that perform less than 200 procedures have a greater than 10% chance of being incorrectly assigned to the worst performing quartile. Minimum surgical volumes and predicted events criteria are required to make evaluating hospitals reliable, and these criteria vary by overall prevalence and between-hospital variability.

High-Dimensional Fixed Effects Profiling Models and Applications in End-Stage Kidney Disease Patients: Current State and Future Directions

Profiling analysis aims to evaluate health care providers, including hospitals, nursing homes, or dialysis facilities among others with respect to a patient outcome, such as 30-day unplanned hospital readmission or mortality. Fixed effects (FE) profiling models have been developed over the last decade, motivated by the overall need to (a) improve accurate identification or "flagging" of under-performing providers, (b) relax assumptions inherent in random effects (RE) profiling models, and (c) take into consideration the unique disease characteristics and care/treatment processes of end-stage kidney disease (ESKD) patients on dialysis. In this paper, we review the current state of FE methodologies and their rationale in the ESKD population and illustrate applications in four key areas: profiling dialysis facilities for (1) patient hospitalizations over time (longitudinally) using standardized dynamic readmission ratio (SDRR), (2) identification of dialysis facility characteristics (e.g., staffing level) that contribute to hospital readmission, and (3) adverse recurrent events using standardized event ratio (SER). Also, we examine the operating characteristics with a focus on FE profiling models. Throughout these areas of applications to the ESKD population, we identify challenges for future research in both methodology and clinical studies.

Updating the model for Risk-Standardizing survival for In-Hospital cardiac arrest to facilitate hospital comparisons

BACKGROUND

Risk-standardized survival rates (RSSR) for in-hospital cardiac arrest (IHCA) have been widely used for hospital benchmarking and research. The novel coronavirus 2019 (COVID-19) pandemic has led to a substantial decline in IHCA survival as COVID-19 infection is associated with markedly lower survival. Therefore, there is a need to update the model for computing RSSRs for IHCA given the COVID-19 pandemic.

METHODS

Within Get With The Guidelines®-Resuscitation, we identified 53,922 adult patients with IHCA from March, 2020 to December, 2021 (the COVID-19 era). Using hierarchical logistic regression, we derived and validated an updated model for survival to hospital discharge and compared the performance of this updated RSSR model with the previous model.

RESULTS

The survival rate was 21.0% and 20.8% for the derivation and validation cohorts, respectively. The model had good discrimination (C-statistic 0.72) and excellent calibration. The updated parsimonious model comprised 13 variables-all 9 predictors in the original model as well as 4 additional predictors, including COVID-19 infection status. When applied to data from the pre-pandemic period of 2018-2019, there was a strong correlation (r = 0.993) between RSSRs obtained from the updated and the previous models.

CONCLUSION

We have derived and validated an updated model to risk-standardize hospital rates of survival for IHCA. The updated model yielded RSSRs that were similar to the initial model for IHCAs in the pre-pandemic period and can be used for supporting ongoing efforts to benchmark hospitals and facilitate research that uses data from either before or after the emergence of COVID-19.

Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data

Background

Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.

Materials and methods

Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.

Results

Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [AUC = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; AUC = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (R ² = 0.242 and R ² = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: ρ = 0, p = 0.99; RSMR: ρ = 0.017, p = 0.56; reliability-adjusted RSMR: ρ = 0.067; p = 0.026).

Conclusion

Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.

Impact of shock aetiology and hospital characteristics on the clinical profile, management and prognosis of patients with non ACS-related cardiogenic shock

BACKGROUND

A significant proportion of cases of cardiogenic shock (CS) are due aetiologies other than acute coronary syndromes (non ACS-CS). We assessed differences regarding clinical profile, management, and prognosis according to the cause of CS among nonselected patients with CS from a large nationwide database.

METHODS

We performed an observational study including patients admitted from the hospitals of the Spanish National Health System (SNHS) with a principal or secondary diagnosis code of CS (2016-2019). Data were obtained from the Minimum Basic Data Set (MBDS). Hospitals were classified according to the availability of cardiology related resources, as well as the availability of Intensive Cardiac Care Unit (ICCU).

RESULTS

A total of 10,826 episodes of CS were included, of whom 5,495 (50.8%) were non-ACS related. Non ACS-CS patients were younger (71.5 vs. 72.4 years) and had a lower burden of arteriosclerosis-related comorbidities. Non ACS-CS cases underwent less often invasive procedures and presented lower in-hospital mortality (57.1% vs. 61%,p < 0.001). The most common main diagnosis among non ACS-CS was acute decompensation of chronic heart failure (ADCHF) (35.4%). A lower risk-adjusted in-hospital mortality rate was observed in high volume hospitals (52.6% vs. 56.7%; p < 0.001), as well as in centers with ICCU (OR: 0.71; CI 95%: 0.58-0.87; p < 0.001).

CONCLUSIONS

More than a half of cases of CS were due to non-ACS causes. Non ACS-CS cases are a very heterogeneous group, with different clinical profile and management. Management at high-volume hospitals and availability of ICCU were associated with lower risk adjusted mortality among non ACS-CS patients.

The Conditional Super Learner

Using cross validation to select the best model from a library is standard practice in machine learning. Similarly, meta learning is a widely used technique where models previously developed are combined (mainly linearly) with the expectation of improving performance with respect to individual models. In this article we consider the Conditional Super Learner (CSL), an algorithm that selects the best model candidate from a library of models conditional on the covariates. The CSL expands the idea of using cross validation to select the best model and merges it with meta learning. We propose an optimization algorithm that finds a local minimum to the problem posed and proves that it converges at a rate faster than O_p(n^-1/4). We offer empirical evidence that: (1) CSL is an excellent candidate to substitute stacking and (2) CLS is suitable for the analysis of Hierarchical problems. Additionally, implications for global interpretability are emphasized.

Analysis of hospital readmissions with competing risks

The 30-day hospital readmission rate has been used in provider profiling for evaluating inter-provider care coordination, medical cost effectiveness, and patient quality of life. Current profiling analyzes use logistic regression to model 30-day readmission as a binary outcome, but one disadvantage of this approach is that this outcome is strongly affected by competing risks (e.g., death). Thus, one, perhaps unintended, consequence is that if two facilities have the same rates of readmission, the one with the higher rate of competing risks will have the lower 30-day readmission rate. We propose a discrete time competing risk model wherein the cause-specific readmission hazard is used to assess provider-level effects. This approach takes account of the timing of events and focuses on the readmission rates which are of primary interest. The quality measure, then is a standardized readmission ratio, akin to a standardized mortality ratio. This measure is not systematically affected by the rate of competing risks. To facilitate the estimation and inference of a large number of provider effects, we develop an efficient Blockwise Inversion Newton algorithm, and a stabilized robust score test that overcomes the conservative nature of the classical robust score test. An application to dialysis patients demonstrates improved profiling, model fitting, and outlier detection over existing methods.

Variation across centers in standardized mortality ratios for congenital diaphragmatic hernia receiving extracorporeal life support

BACKGROUND

We sought to elucidate the degree of variation across centers by calculating center-specific standardized mortality ratios (SMRs) for infants with congenital diaphragmatic hernia (CDH) requiring extracorporeal life support (ECLS).

METHODS

The Extracorporeal Life Support Organization (ELSO) registry data (2000-2019) were used to estimate SMRs. Center-specific SMRs and their 95% confidence intervals (CIs) were used to identify centers with mortality as significantly worse (SW), significantly better (SB), or not different (ND) relative to the median standardized mortality rate.

RESULTS

We identified 4,223 neonates with CDH from 109 centers. SMRs were risk-adjusted for pre-ECLS case-mix (birthweight, sex, race, 5 min Apgar, blood gases, gestational age, hernia side, prenatal diagnosis, pre-ECLS arrest, and comorbidities). Observed (unadjusted) mortality rates across centers varied substantially (range: 14.3%-90.9%; interquartile range [IQR]: 42.9%-62.1%). Thirteen centers (11.9%) had SB SMRs< 1 (SMR 0.52 to 0.84), 7 centers (6.4%) had SW SMRs>1 (SMR 1.25 to 1.43), and 89 centers (81.7%) had SMRs ND relative to the median SMR rate across all centers (i.e., SMR not different from one). Descriptive analyses demonstrated that SB centers had a lower proportion of cases with renal complications, infectious complications and discontinuation of ECLS owing to complications, as well as differences in pre-ECLS treatments and timing of CDH repair, compared to SW and ND centers.

CONCLUSION

This study specifically identified ECLS centers with higher and lower survival for patients with CDH, which may serve as a benchmark for institutional quality improvement. Future studies are needed to identify those specific processes at those centers that leads to favorable outcomes with the goal of improving care globally.

LEVEL OF EVIDENCE

Level III.

MEASURING PERFORMANCE FOR END-OF-LIFE CARE

Although not without controversy, readmission is entrenched as a hospital quality metric with statistical analyses generally based on fitting a logistic-Normal generalized linear mixed model. Such analyses, however, ignore death as a competing risk, although doing so for clinical conditions with high mortality can have profound effects; a hospital's seemingly good performance for readmission may be an artifact of it having poor performance for mortality. in this paper we propose novel multivariate hospital-level performance measures for readmission and mortality that derive from framing the analysis as one of cluster-correlated semi-competing risks data. We also consider a number of profiling-related goals, including the identification of extreme performers and a bivariate classification of whether the hospital has higher-/lower-than-expected readmission and mortality rates via a Bayesian decision-theoretic approach that characterizes hospitals on the basis of minimizing the posterior expected loss for an appropriate loss function. in some settings, particularly if the number of hospitals is large, the computational burden may be prohibitive. To resolve this, we propose a series of analysis strategies that will be useful in practice. Throughout, the methods are illustrated with data from CMS on N = 17,685 patients diagnosed with pancreatic cancer between 2000-2012 at one of J = 264 hospitals in California.

Do all Patients Get the Same Care Across Hospitals?

There is extensive research demonstrating significant variation in the utilization of surgery and outcomes from surgery, including differences in mortality, complications, readmission, and failure to rescue. Literature suggests that these variations exist across as well as within small area geographies in the United States. There is also significant evidence of variation in access and outcomes from surgery that is attributable to race. Emerging research is demonstrating that there may be some variation attributable to a patient's social determinants of health and their lived averment. Those affected must work together to determine rate of utilization and how much variation is acceptable.

Improving large-scale estimation and inference for profiling health care providers

Provider profiling has been recognized as a useful tool in monitoring health care quality, facilitating inter-provider care coordination, and improving medical cost-effectiveness. Existing methods often use generalized linear models with fixed provider effects, especially when profiling dialysis facilities. As the number of providers under evaluation escalates, the computational burden becomes formidable even for specially designed workstations. To address this challenge, we introduce a serial blockwise inversion Newton algorithm exploiting the block structure of the information matrix. A shared-memory divide-and-conquer algorithm is proposed to further boost computational efficiency. In addition to the computational challenge, the current literature lacks an appropriate inferential approach to detecting providers with outlying performance especially when small providers with extreme outcomes are present. In this context, traditional score and Wald tests relying on large-sample distributions of the test statistics lead to inaccurate approximations of the small-sample properties. In light of the inferential issue, we develop an exact test of provider effects using exact finite-sample distributions, with the Poisson-binomial distribution as a special case when the outcome is binary. Simulation analyses demonstrate improved estimation and inference over existing methods. The proposed methods are applied to profiling dialysis facilities based on emergency department encounters using a dialysis patient database from the Centers for Medicare & Medicaid Services.

Variation in Out-of-Hospital Cardiac Arrest Survival Across Emergency Medical Service Agencies

BACKGROUND

Although studies have reported variation in out-of-hospital cardiac arrest (OHCA) survival by geographic location, little is known about variation in OHCA survival at the level of emergency medical service (EMS) agencies-which may have modifiable practices, unlike counties and regions. We quantified the variation in OHCA survival across EMS agencies and explored whether variation in 2 specific EMS resuscitation practices were associated with survival to hospital admission.

METHODS

Within the Cardiac Arrest Registry to Enhance Survival, a prospective registry representing ≈51% of the US population, we identified 258 342 OHCAs from 764 EMS agencies with >10 OHCA cases annually during 2015 to 2019. Using hierarchical logistic regression, risk-standardized rates of survival to hospital admission were computed for each EMS agency. We quantified inter-agency variation in survival with median odds ratios and assessed the association of 2 resuscitation practices (EMS response time and the proportion of OHCAs with termination of resuscitation without meeting futility criteria) with EMS agency survival rates to hospital admission.

RESULTS

Across 764 EMS agencies comprising 258 342 OHCAs, the median risk-standardized rate of survival to hospital admission was 27.3% (interquartile range, 24.5%-30.1%; range: 16.0%-45.6%). The adjusted median odds ratio was 1.35 (95% CI, 1.32-1.39), denoting that the odds of survival of 2 patients with identical covariates varied by 35% at 2 randomly selected EMS agencies. EMS agencies in the lowest quartile of risk-standardized survival had longer EMS response times when compared with the highest quartile (12.0±3.4 versus 9.0±2.6 minutes; P<0.001), and a higher proportion of OHCAs with termination of resuscitation without meeting futility criteria (27.9±16.1% versus 18.9±11.4%; P<0.001).

CONCLUSIONS

Survival after OHCA varies widely across EMS agencies. EMS response times and termination of resuscitation practices were associated with agency-level rates of survival to hospital admission, suggesting potentially modifiable practices which can improve OHCA survival.

Statistical Approaches for Quantifying the Quality of Neurosurgical Care

BACKGROUND

Quantifying quality of health care can provide valuable information to patients, providers, and policy makers. However, the observational nature of measuring quality complicates assessments.

METHODS

We describe a conceptual model for defining quality and its implications about the data collected, how to make inferences about quality, and the assumptions required to provide statistically valid estimates. Twenty-one binary or polytomous quality measures collected from 101,051 adult Medicaid beneficiaries aged 18-64 years with schizophrenia from 5 U.S. states show methodology. A categorical principal components analysis establishes dimensionality of quality, and item response theory models characterize the relationship between each quality measure and a unidimensional quality construct. Latent regression models estimate racial/ethnic and geographic quality disparities.

RESULTS

More than 90% of beneficiaries filled at least 1 antipsychotic prescription and 19% were hospitalized for schizophrenia during a 12-month observational period in our multistate cohort with approximately 2/3 nonwhite beneficiaries. Four quality constructs emerged: inpatient, emergency room, pharmacologic/ambulatory, and ambulatory only. Using a 2-parameter logistic model, pharmacologic/ambulatory care quality varied from -2.35 to 1.26 (higher = better quality). Black and Latinx beneficiaries had lower pharmacologic/ambulatory quality compared with whites. Race/ethnicity modified the association of state and pharmacologic/ambulatory care quality in latent regression modeling. Average quality ranged from -0.28 (95% confidence interval, -2.15 to 1.04) for blacks in New Jersey to 0.46 [95% confidence interval, -0.89 to 1.40] for whites in Michigan.

CONCLUSIONS

By combining multiple quality measures using item response theory models, a composite measure can be estimated that has more statistical power to detect differences among subjects than the observed mean per subject.

Patient-reported outcomes as hospital performance measures: the challenge of confounding and how to handle it

It is highly appealing to use patient-reported outcomes (PROs) as hospital performance measures; however, so far, the attention to key methodological issues has been limited. One of the most critical challenges when comparing PRO-based performance measures across providers is to rule out confounding. In this paper, we explain confounding and why it matters when comparing across providers. Using examples from studies, we present potential strategies for dealing with confounding when using PRO data at an aggregated level. We aim to give clinicians an overview of how confounding can be addressed in both the design stage (restriction, matching, self-controlled design and propensity score) and the analysis stage (stratification, standardization and multivariable adjustment, including multilevel analysis) of a study. We also briefly discuss strategies for confounding control when data on important confounders are missing or unavailable.

Accounting for total variation and robustness in profiling health care providers

Monitoring outcomes of health care providers, such as patient deaths, hospitalizations, and hospital readmissions, helps in assessing the quality of health care. We consider a large database on patients being treated at dialysis facilities in the United States, and the problem of identifying facilities with outcomes that are better than or worse than expected. Analyses of such data have been commonly based on random or fixed facility effects, which have shortcomings that can lead to unfair assessments. A primary issue is that they do not appropriately account for variation between providers that is outside the providers' control due, for example, to unobserved patient characteristics that vary between providers. In this article, we propose a smoothed empirical null approach that accounts for the total variation and adapts to different provider sizes. The linear model provides an illustration that extends easily to other non-linear models for survival or binary outcomes, for example. The empirical null method is generalized to allow for some variation being due to quality of care. These methods are examined with numerical simulations and applied to the monitoring of survival in the dialysis facility data.

Thirteen-year trends in hospitalization and outcomes of patients with heart failure in Spain

BACKGROUND

Heart failure is one of the most pressing current public health concerns. However, in Spain there is a lack of population data. We aimed to examine thirteen-year nationwide trends in heart failure hospitalization, in-hospital mortality and 30-day readmission rates in Spain.

METHODS

We conducted a retrospective observational study of patients discharged with the principal diagnosis of heart failure from The National Health System' acute hospitals during 2003-2015. The source of the data was the Minimum Basic Data Set. Temporal trends were modelled using Poisson regression analysis. The risk-standardized in-hospital mortality ratio was calculated using a multilevel risk adjustment logistic regression model.

RESULTS

A total of 1 254 830 episodes of heart failure were selected. Throughout 2003-2015, the number of hospital discharges with principal diagnosis of heart failure increased by 61%. Discharge rates weighted by age and sex increased during the period [incidence rate ratio (IRR): 1.03; 95% confidence interval (95% CI): 1.03-1.03; P < .001)], although this increase was motivated by the increase in older age groups (≥75 years old). The crude mortality rate diminished (IRR: 0.99; 95% CI: 0.98-1, P < .001), but 30-day readmission rate increased (IRR: 1.05; 95% CI: 1.04-1.06; P < .001). The risk-standardized in-hospital mortality ratio did not change throughout the study period (IRR: 0.997; 95% CI: 0.992-1; P = .32).

CONCLUSIONS

From 2003 to 2015, heart failure admission rates increased significantly in Spain as a consequence of the sustained increase of hospitalization in the population ≥75 years. 30-day readmission rates increased, but the risk-standardized in-hospital mortality ratio did not significantly change for the same period.

Predicting In-Hospital Mortality in Patients Undergoing Percutaneous Coronary Intervention

BACKGROUND

Standardization of risk is critical in benchmarking and quality improvement efforts for percutaneous coronary interventions (PCIs). In 2018, the CathPCI Registry was updated to include additional variables to better classify higher-risk patients.

OBJECTIVES

This study sought to develop a model for predicting in-hospital mortality risk following PCI incorporating these additional variables.

METHODS

Data from 706,263 PCIs performed between July 2018 and June 2019 at 1,608 sites were used to develop and validate a new full and pre-catheterization model to predict in-hospital mortality, and a simplified bedside risk score. The sample was randomly split into a development cohort (70%, n = 495,005) and a validation cohort (30%, n = 211,258). The authors created 1,000 bootstrapped samples of the development cohort and used stepwise selection logistic regression on each sample. The final model included variables that were selected in at least 70% of the bootstrapped samples and those identified a priori due to clinical relevance.

RESULTS

In-hospital mortality following PCI varied based on clinical presentation. Procedural urgency, cardiovascular instability, and level of consciousness after cardiac arrest were most predictive of in-hospital mortality. The full model performed well, with excellent discrimination (C-index: 0.943) in the validation cohort and good calibration across different clinical and procedural risk cohorts. The median hospital risk-standardized mortality rate was 1.9% and ranged from 1.1% to 3.3% (interquartile range: 1.7% to 2.1%).

CONCLUSIONS

The risk of mortality following PCI can be predicted in contemporary practice by incorporating variables that reflect clinical acuity. This model, which includes data previously not captured, is a valid instrument for risk stratification and for quality improvement efforts.

Incidence and outcomes of hospital treated acute myocarditis from 2003 to 2015 in Spain

BACKGROUND

There are no data on population-based epidemiological changes in acute myocarditis in Europe. Our aim was to evaluate temporal trends in incidence, clinical features and outcomes of hospital treated acute myocarditis (AM) in Spain from 2003 to 2015.

METHODS

We conducted a retrospective longitudinal study using information of all hospital discharges of the Spanish National Health System. All episodes with a discharge diagnosis of AM from 1 January 2003 to 31 December 2015 were included. The risk-standardized in-hospital mortality ratio (RSMR) was calculated using a multilevel risk-adjustment model developed by the Medicare and Medicaid Services. Temporal trends for in-hospital mortality were modelled using Poisson regression analysis.

RESULTS

A total of 11 147 episodes of AM were analysed, most of them idiopathic (94.7%). The rate of AM discharges increased along the period, from 13 to 30/million inhabitants/year (2003-2015), and this increase was statistically significant when weighted by age and sex (incidence rate ratio, IRR 1.06, 95% CI 1.04-1.08, P = .001). In-hospital crude mortality rate was 3.1%, diminishing significantly along 2003-2015 (IRR 0.95, 95% CI 0.92-0.99, P = .02). RSMR also significantly diminished along the period (IRR 0.95, 95% CI 0.92-0.99, P = .01). Renal failure (OR 7.03, 5.38-9.18, P = .001), liver disease (OR 4.61, 2.59-8.21, P = .001), pneumonia (OR 4.13, 2.75-6.20, P = .001) and heart failure (OR 1.91, 95% CI 1.47-2.47, P = .001) were the strongest independent factors associated with in-hospital mortality.

CONCLUSIONS

Acute myocarditis is an uncommon entity, although hospital discharges have increased in Spain along the study period. Most of AM were idiopathic. Adjusted mortality was low and seemed to decrease from 2003 to 2015, suggesting an improvement in AM management.

Impact of Heart Failure on In-Hospital Outcomes after Surgical Femoral Neck Fracture Treatment

BACKGROUND

Femoral neck fracture (FNF) is a common condition with a rising incidence, partly due to aging of the population. It is recommended that FNF should be treated at the earliest opportunity, during daytime hours, including weekends. However, early surgery shortens the available time for preoperative medical examination. Cardiac evaluation is critical for good surgical outcomes as most of these patients are older and frail with other comorbid conditions, such as heart failure. The aim of this study was to determine the impact of heart failure on in-hospital outcomes after surgical femoral neck fracture treatment.

METHODS

We performed a retrospective study using the Spanish National Hospital Discharge Database, 2007-2015. We included patients older than 64 years treated for reduction and internal fixation of FNF. Demographic characteristics of patients, as well as administrative variables, related to patient's diseases and procedures performed during the episode were evaluated.

RESULTS

A total of 234,159 episodes with FNF reduction and internal fixation were identified from Spanish National Health System hospitals during the study period; 986 (0.42%) episodes were excluded, resulting in a final study population of 233,173 episodes. Mean age was 83.7 (±7) years and 179,949 (77.2%) were women (p < 0.001). In the sample, 13,417 (5.8%) episodes had a main or secondary diagnosis of heart failure (HF) (p < 0.001). HF patients had a mean age of 86.1 (±6.3) years, significantly older than the rest (p < 0.001). All the major complications studied showed a higher incidence in patients with HF (p < 0.001). Unadjusted in-hospital mortality was 4.1%, which was significantly higher in patients with HF (18.2%) compared to those without HF (3.3%) (p < 0.001). The average length of stay (LOS) was 11.9 (±9.1) and was also significantly higher in the group with HF (16.5 ± 13.1 vs. 11.6 ± 8.7; p < 0.001).

CONCLUSIONS

Patients with HF undergoing FNF surgery have longer length of stay and higher rates of both major complications and mortality than those without HF. Although their average length of stay has decreased in the last few years, their mortality rate has remained unchanged.

Correlation between compensated patient claims and 30-day mortality

OBJECTIVE

To test if compensation claims from patients (reported to the Norwegian System of Patient Injury Compensation) are correlated with the existing quality indicator of 30-day mortality (based on data from Norwegian Patient Registry). This correlation has not been previously evaluated.

DESIGN

The association between patient claims and 30-day mortality at hospital trust level was assessed by the Pearson correlation coefficient.

SETTING

The Norwegian System of Patient Injury Compensation is a governmental agency under the Ministry of Health and Care Services and deals with patient-reported complaints about incorrect treatment in the public and private healthcare services. Patient-reported claims may be an indicator of healthcare quality, as 30-day mortality.

PARTICIPANTS

All 19 Norwegian hospital trusts.

INTERVENTIONS

: None.

MAIN OUTCOME MEASURE

Patient claims rates, 30-day mortality and Pearson correlation coefficient.

RESULTS

Both number of deaths within 30 days and number of claims have declined over time. High correlation (0.77, P < 0.001) was found between number of deaths within 30 days and the total number of claims. In addition, an even stronger association was found with approved claims, with a correlation coefficient of 0.83 (P < 0.001). Moreover, adjusted 30-day mortality was significantly correlated with the patient-claim rate using number of bed-days as denominator, but not when using number of discharges.

CONCLUSIONS

The results from the present study indicate an association between compensation claims from patients and 30-day mortality, suggesting that both parameters reflect the latent quality of care for the hospital trusts, but they may capture different aspects of care.

Assessing Nursing Homes Quality Indicators' Between-Provider Variability and Reliability: A Cross-Sectional Study Using ICCs and Rankability

Nursing home quality indicators are often used to publicly report the quality of nursing home care. In Switzerland, six national nursing home quality indicators covering four clinical domains (polypharmacy, pain, use of physical restraints and weight loss) were recently developed. To allow for meaningful comparisons, these indicators must reliably show differences in quality of care levels between nursing homes. This study’s objectives were to assess nursing home quality indicators’ between-provider variability and reliability using intraclass correlations and rankability. This approach has not yet been used in long-term care contexts but presents methodological advantages. This cross-sectional multicenter study uses data of 11,412 residents from a convenience sample of 152 Swiss nursing homes. After calculating intraclass correlation 1 (ICC1) and rankability, we describe between-provider variability for each quality indicator using empirical Bayes estimate-based caterpillar plots. To assess reliability, we used intraclass correlation 2 (ICC2). Overall, ICC1 values were high, ranging from 0.068 (95% confidence interval (CI) 0.047–0.086) for polypharmacy to 0.396 (95% CI 0.297–0.474) for physical restraints, with quality indicator caterpillar plots showing sufficient between-provider variability. However, testing for rankability produced mixed results, with low figures for two indicators (0.144 for polypharmacy; 0.471 for self-reported pain) and moderate to high figures for the four others (from 0.692 for observed pain to 0.976 for physical restraints). High ICC2 figures, ranging from 0.896 (95% CI 0.852–0.917) (self-reported pain) to 0.990 (95% CI 0.985–0.993) (physical restraints), indicated good reliability for all six quality indicators. Intraclass correlations and rankability can be used to assess nursing home quality indicators’ between-provider variability and reliability. The six selected quality indicators reliably distinguish care differences between nursing homes and can be recommended for use, although the variability of two—polypharmacy and self-reported pain—is substantially chance-driven, limiting their utility.

Examining Hospital Variation on Multiple Indicators of Stroke Quality of Care

BACKGROUND

Provider profiling involves comparing the performance of hospitals on indicators of quality of care. Typically, provider profiling examines the performance of hospitals on each quality indicator in isolation. Consequently, one cannot formally examine whether hospitals that have poor performance on one indicator also have poor performance on a second indicator.

METHODS

We used Bayesian multivariate response random effects logistic regression model to simultaneously examine variation and covariation in multiple binary indicators across hospitals. We considered 7 binary patient-level indicators of quality of care for patients presenting to hospital with a diagnosis of acute stroke. We examined between-hospital variation in these 7 indicators across 86 hospitals in Ontario, Canada.

RESULTS

The number of patients eligible for each indicator ranged from 1321 to 14 079. There were 7 pairs of indicators for which there was a strong correlation between a hospital's performance on each of the 2 indicators. Twenty-nine of the 86 hospitals had a probability higher than 0.90 of having worse performance than average on at least 4 of the 7 indicators. Seven of the 86 of hospitals had a probability higher than 0.90 of having worse performance than average on at least 5 indicators. Fourteen of the 86 of hospitals had a probability higher than 0.50 of having worse performance than average on at least 6 indicators. No hospitals had a probability higher than 0.50 of having worse performance than average on all 7 indicators.

CONCLUSIONS

These findings suggest that there are a small number of hospitals that perform poorly on at least half of the quality indicators, and that certain indicators tend to cluster together. The described methods allow for targeting quality improvement initiatives at these hospitals.

Impact of sex differences and network systems on the in-hospital mortality of patients with ST-segment elevation acute myocardial infarction

INTRODUCTION AND OBJECTIVES

Network systems have achieved reductions in both time to reperfusion and in-hospital mortality in patients with ST-segment elevation myocardial infarction (STEMI). However, the data have not been disaggregated by sex. The aim of this study was to analyze the influence of network systems on sex differences in primary percutaneous coronary intervention (pPCI) and in-hospital mortality from 2005 to 2015.

METHODS

The Minimum Data Set of the Spanish National Health System was used to identify patients with STEMI. Logistic multilevel regression models and Poisson regression analysis were used to calculate risk-standardized in-hospital mortality ratios and incidence rate ratios (IRRs).

RESULTS

Of 324 998 STEMI patients, 277 281 were selected after exclusions (29% women). Even when STEMI networks were established, the use of reperfusion therapy (PCI, fibrinolysis, and CABG) was lower in women than in men from 2005 to 2015: 56.6% vs 75.6% in men and 36.4% vs 57.0% in women, respectively (both P<.001). pPCI use increased from 34.9% to 68.1% in men (IRR, 1.07) and from 21.7% to 51.7% in women (IRR, 1.08). The crude in-hospital mortality rate was higher in women (9.3% vs 18.7%; P<.001) but decreased from 2005 to 2015 (IRRs, 0.97 for men and 0.98 for women; both P < .001). Female sex was an independent risk factor for mortality (adjusted OR, 1.23; P<.001). The risk-standardized in-hospital mortality ratio was lower in women when STEMI networks were in place (16.9% vs 19.1%, P<.001). pPCI and the presence of STEMI networks were associated with lower in-hospital mortality in women (adjusted ORs, 0.30 and 0.75, respectively; both P<.001).

CONCLUSIONS

Women were less likely to receive pPCI and had higher in-hospital mortality than men throughout the 11-year study period, even with the presence of a network system for STEMI.

Identifying Performance Outliers for Stroke Care Based on Composite Score of Process Indicators: an Observational Study in China

BACKGROUND

Variability in the quality of stroke care is widespread. Identifying performance-based outlier hospitals based on quality indicators (QIs) has become a common practice.

OBJECTIVES

To develop a tool for identifying performance-based outlier hospitals based on risk-adjusted adherence rates of process indicators.

DESIGN

Hospitals were classified into five-level outliers based on the observed-to-expected ratio and P value. The composite quality score was derived by summation of the points for each indicator for each hospital, and associations between outlier status and outcomes were determined.

PARTICIPANTS

Patients diagnosed with acute ischemic stroke, January 1, 2011-May 31, 2017.

INTERVENTION

N/A MAIN OUTCOME MEASURES: Independence at discharge (the modified Rankin Scale = 0-2).

KEY RESULTS

A total of 501,132 patients from 519 hospitals were identified. From 0.39 to 19.65% of hospitals were identified as high outliers according to various QIs. Composite quality scores ranged from - 20 to 16. Providers that were high outliers based on QI2, QI8, QI9, and QI11 had higher independent rates. For composite quality score, each point increase corresponded to an 8% increase in the odds of independent rate.

CONCLUSION

Nationwide variation in the quality of acute stroke care exists at the hospital level. Variability in the quality of stroke care can be captured by our proposed quality score. Applying this quality score as a benchmarking tool could provide audit-level feedback to policymakers and hospitals to aid quality improvement.

A Bayesian hierarchical model for characterizing the diffusion of new antipsychotic drugs

New prescription medications are a primary driver of spending growth in the United States. For patients with severe mental illnesses, second-generation antipsychotic (SGA) medications feature prominently. However, many SGAs are costly, particularly before generic entry, and some may increase the risk of diabetes. Because physicians play a prominent role in new prescription adoption, understanding their prescribing behaviors is policy-relevant. Several features of prescription data, such as different antipsychotic choice sets over time, variable physician prescription volumes, and correlation among drug choices within physicians, complicate inferences. We propose a multivariate Bayesian hierarchical model with piecewise random effects to characterize the diffusion of new antipsychotic drugs. This model captures the complex prescriber-specific relationships among the different diffusion processes and takes advantage of the Bayesian paradigm to quantify uncertainty for all parameters straightforwardly. To evaluate the prescribing patterns for each physician, we propose various indices to identify early new SGA adopters. A sample of nearly 17,000 US physicians whose antipsychotic drug prescribing information was collected between January 1, 1997 and December 31, 2007 illustrates the methods. Determinants of high prescription rates and adoption speeds of new SGAs included physician sex, age, hospital affiliation, physician specialty, and office location. Large within- and between-provider variations in prescribing patterns of new SGAs were identified. Early adopters for one drug were not early adopters for another drug.

Bayesian profiling for cost with zeros to decompose total cost into probability of cost and mean nonzero cost

Cost of health care can vary substantially across hospitals, centers, or providers. Data from electronic health records provide information for studying patterns of cost variation and identifying high or low cost centers. Cost data often include zero values when patients receive no care, and joint two-part models have been developed for clustered cost data with zeros. Standard methods for center comparisons, sometimes called profiling, can use these methods to incorporate zero values into total cost. However, zero costs also provide opportunities to further examine sources of cost variation and outliers. For example, a hospital may have high (or low) cost due to frequency of nonzero cost, amount of nonzero cost, or a combination of those. We give methods for decomposing hospital differences in total cost with zeros into components for probability of use (i.e., of nonzero cost) and for cost of use (mean of nonzero cost). The components multiply to total cost and quantify components on the same easily interpreted multiplicative scales. The methods are based on Bayesian hierarchical models and counterfactual arguments, with Markov chain Monte Carlo estimation. We used simulated data to illustrate use, interpretation, and visualization of the methods in diverse situations, and applied the methods to 30,024 patients at 57 US Veterans Administration hospitals to characterize outlier hospitals in one year cost of inpatient care following a cardiac procedure. Twenty eight percent of patients had zero cost. These methods are useful in providing insight into cost variation and outliers for planning future studies or interventions.

Differences in in-hospital mortality after STEMI versus NSTEMI by sex. Eleven-year trend in the Spanish National Health Service

INTRODUCTION AND OBJECTIVES

Conflicting results have been reported on the possible existence of sex differences in mortality after myocardial infarction (MI). There is also a scarcity of data on the impact of sex on outcomes after ST-segment elevation myocardial infarction (STEMI) and non-STEMI (NSTEMI). The aim of this study was to analyze sex difference trends in sex-related differences in mortality for STEMI and NSTEMI.

METHODS

A retrospective analysis of 445 145 episodes of MI (2005-2015) was carried out using information from the Spanish National Health System. The incidence rates were expressed as events per 10 000 person-years. The denominators (age-specific groups) were obtained from the nationwide census. We calculated crude and adjusted (multilevel logistic regression) mortality. Poisson regression analysis was used to study temporal trends for in-hospital mortality.

RESULTS

A total of 69.8% episodes occurred in men. The mean age in men was 66.1±13.3 years, which was significantly younger than in women, 74.9±12.1 (P<.001). A total of 272 407 (61.2%) episodes were STEMI, and 172 738 (38.8%) were NSTEMI. Women accounted for 28.8% of STEMI and 33.9% of NSTEMI episodes (P <.001). The effect of female sex on risk-adjusted models for in-hospital mortality was the opposite in STEMI (OR for women, 1.18; 95%CI, 1.14-1.22; P <.001) and NSTEMI (OR for women, 0.85; 95%CI, 0.81-0.89; P <.001). MI hospitalization rates were higher in men than in women for all age groups [20 vs 7.7 per 10 000 individuals aged 35-94 years (P <.001)], with a trend to diminish in both sexes.

CONCLUSIONS

Women had a slight but significantly increased risk of in-hospital mortality after MI, but the effect of sex depended on MI type, with women exhibiting higher mortality for STEMI and lower mortality for NSTEMI.

Evaluating the effect of healthcare providers on the clinical path of heart failure patients through a semi-Markov, multi-state model

Background

Investigating similarities and differences among healthcare providers, on the basis of patient healthcare experience, is of interest for policy making. Availability of high quality, routine health databases allows a more detailed analysis of performance across multiple outcomes, but requires appropriate statistical methodology.

Methods

Motivated by analysis of a clinical administrative database of 42,871 Heart Failure patients, we develop a semi-Markov, illness-death, multi-state model of repeated admissions to hospital, subsequent discharge and death. Transition times between these health states each have a flexible baseline hazard, with proportional hazards for patient characteristics (case-mix adjustment) and a discrete distribution for frailty terms representing clusters of providers. Models were estimated using an Expectation-Maximization algorithm and the number of clusters was based on the Bayesian Information Criterion.

Results

We are able to identify clusters of providers for each transition, via the inclusion of a nonparametric discrete frailty. Specifically, we detect 5 latent populations (clusters of providers) for the discharge transition, 3 for the in-hospital to death transition and 4 for the readmission transition. Out of hospital death rates are similar across all providers in this dataset. Adjusting for case-mix, we could detect those providers that show extreme behaviour patterns across different transitions (readmission, discharge and death).

Conclusions

The proposed statistical method incorporates both multiple time-to-event outcomes and identification of clusters of providers with extreme behaviour simultaneously. In this way, the whole patient pathway can be considered, which should help healthcare managers to make a more comprehensive assessment of performance.

Assessing the Impacts of Misclassified Case-Mix Factors on Health Care Provider Profiling: Performance of Dialysis Facilities

Quantitative metrics are used to develop profiles of health care institutions, including hospitals, nursing homes, and dialysis clinics. These profiles serve as measures of quality of care, which are used to compare institutions and determine reimbursement, as a part of a national effort led by the Center for Medicare and Medicaid Services in the United States. However, there is some concern about how misclassification in case-mix factors, which are typically accounted for in profiling, impacts results. We evaluated the potential effect of misclassification on profiling results, using 20 744 patients from 2740 dialysis facilities in the US Renal Data System. In this case study, we compared 30-day readmission as the profiling outcome measure, using comorbidity data from either the Center for Medicare and Medicaid Services Medical Evidence Report (error-prone) or Medicare claims (more accurate). Although the regression coefficient of the error-prone covariate demonstrated notable bias in simulation, the outcome measure—standardized readmission ratio—and profiling results were quite robust; for example, correlation coefficient of 0.99 in standardized readmission ratio estimates. Thus, we conclude that misclassification on case-mix did not meaningfully impact overall profiling results. We also identified both extreme degree of case-mix factor misclassification and magnitude of between-provider variability as 2 factors that can potentially exert enough influence on profile status to move a clinic from one performance category to another (eg, normal to worse performer).

Profiling dialysis facilities for adverse recurrent events

Profiling analysis aims to evaluate health care providers, such as hospitals, nursing homes, or dialysis facilities, with respect to a patient outcome. Previous profiling methods have considered binary outcomes, such as 30-day hospital readmission or mortality. For the unique population of dialysis patients, regular blood works are required to evaluate effectiveness of treatment and avoid adverse events, including dialysis inadequacy, imbalance mineral levels, and anemia among others. For example, anemic events (when hemoglobin levels exceed normative range) are recurrent and common for patients on dialysis. Thus, we propose high-dimensional Poisson and negative binomial regression models for rate/count outcomes and introduce a standardized event ratio measure to compare the event rate at a specific facility relative to a chosen normative standard, typically defined as an "average" national rate across all facilities. Our proposed estimation and inference procedures overcome the challenge of high-dimensional parameters for thousands of dialysis facilities. Also, we investigate how overdispersion affects inference in the context of profiling analysis. The proposed methods are illustrated with profiling dialysis facilities for recurrent anemia events.