Can we trust the standardized mortality ratio? A formal analysis and evaluation based on axiomatic requirements

Regional differences in infective endocarditis epidemiology and outcomes in Spain. A contemporary population-based study

INTRODUCTION AND OBJECTIVES

Our aim was to describe the contemporary epidemiological profile of infective endocarditis (IE) in Spain, and to evaluate variations in IE incidence, characteristics, and outcomes among the different Spanish regions (autonomous communities [AC]).

METHODS

We conducted a retrospective, population-based study, using data obtained from national in-patient hospital activity of all patients discharged with a diagnosis of IE from hospitals included in the Spanish National Health System, from January 2016 to December 2019. Differences in the IE profile between the 17 Spanish AC were analyzed.

RESULTS

A total of 9008 hospitalization episodes were identified during the study period. Standardized incidence of IE was 5.77 (95%CI, 5.12-6.41) cases per 100 000 population. Regarding predisposing conditions, 26.8% of episodes occurred in prosthetic valve carriers, 36.8% had some kind of valve heart disease, and 10.6% had a cardiac implantable electronic device. Significant differences were found between AC in terms of incidence, predisposing conditions, and microbiological profile. Cardiac surgery was performed in 19.3% of episodes in the total cohort, and in 33.4% of the episodes treated in high-volume referral centers, with wide variations among AC. Overall in-hospital mortality was 27.2%. Risk-adjusted mortality rates also varied significantly among regions.

CONCLUSIONS

We found wide heterogeneity among Spanish AC in terms of incidence rates and the clinical and microbiological characteristics of IE episodes. The proportion of patients undergoing surgery was low and in-hospital mortality rates were high, with wide differences among regions. The development of regional networks with referral centers for IE could facilitate early surgery and improve outcomes.

Prognostic score-based methods for estimating center effects based on survival probability: Application to post-kidney transplant survival

In evaluating the performance of different facilities or centers on survival outcomes, the standardized mortality ratio (SMR), which compares the observed to expected mortality has been widely used, particularly in the evaluation of kidney transplant centers. Despite its utility, the SMR may exaggerate center effects in settings where survival probability is relatively high. An example is one-year graft survival among U.S. kidney transplant recipients. We propose a novel approach to estimate center effects in terms of differences in survival probability (ie, each center versus a reference population). An essential component of the method is a prognostic score weighting technique, which permits accurately evaluating centers without necessarily specifying a correct survival model. Advantages of our approach over existing facility-profiling methods include a metric based on survival probability (greater clinical relevance than ratios of counts/rates); direct standardization (valid to compare between centers, unlike indirect standardization based methods, such as the SMR); and less reliance on correct model specification (since the assumed model is used to generate risk classes as opposed to fitted-value based 'expected' counts). We establish the asymptotic properties of the proposed weighted estimator and evaluate its finite-sample performance under a diverse set of simulation settings. The method is then applied to evaluate U.S. kidney transplant centers with respect to graft survival probability.

Models and methods for analysing clustered recurrent hospitalisations in the presence of COVID-19 effects

Recurrent events such as hospitalisations are outcomes that can be used to monitor dialysis facilities' quality of care. However, current methods are not adequate to analyse data from many facilities with multiple hospitalisations, especially when adjustments are needed for multiple time scales. It is also controversial whether direct or indirect standardisation should be used in comparing facilities. This study is motivated by the need of the Centers for Medicare and Medicaid Services to evaluate US dialysis facilities using Medicare claims, which involve almost 8,000 facilities and over 500,000 dialysis patients. This scope is challenging for current statistical software's computational power. We propose a method that has a flexible baseline rate function and is computationally efficient. Additionally, the proposed method shares advantages of both indirect and direct standardisation. The method is evaluated under a range of simulation settings and demonstrates substantially improved computational efficiency over the existing R package survival. Finally, we illustrate the method with an important application to monitoring dialysis facilities in the U.S., while making time-dependent adjustments for the effects of COVID-19.

Developing and externally validating a machine learning risk prediction model for 30-day mortality after stroke using national stroke registers in the UK and Sweden

OBJECTIVES

We aimed to develop and externally validate a generalisable risk prediction model for 30-day stroke mortality suitable for supporting quality improvement analytics in stroke care using large nationwide stroke registers in the UK and Sweden.

DESIGN

Registry-based cohort study.

SETTING

Stroke registries including the Sentinel Stroke National Audit Programme (SSNAP) in England, Wales and Northern Ireland (2013-2019) and the national Swedish stroke register (Riksstroke 2015-2020).

PARTICIPANTS AND METHODS

Data from SSNAP were used for developing and temporally validating the model, and data from Riksstroke were used for external validation. Models were developed with the variables available in both registries using logistic regression (LR), LR with elastic net and interaction terms and eXtreme Gradient Boosting (XGBoost). Performances were evaluated with discrimination, calibration and decision curves.

OUTCOME MEASURES

The primary outcome was all-cause 30-day in-hospital mortality after stroke.

RESULTS

In total, 488 497 patients who had a stroke with 12.4% 30-day in-hospital mortality were used for developing and temporally validating the model in the UK. A total of 128 360 patients who had a stroke with 10.8% 30-day in-hospital mortality and 13.1% all mortality were used for external validation in Sweden. In the SSNAP temporal validation set, the final XGBoost model achieved the highest area under the receiver operating characteristic curve (AUC) (0.852 (95% CI 0.848 to 0.855)) and was well calibrated. The performances on the external validation in Riksstroke were as good and achieved AUC at 0.861 (95% CI 0.858 to 0.865) for in-hospital mortality. For Riksstroke, the models slightly overestimated the risk for in-hospital mortality, while they were better calibrated at the risk for all mortality.

CONCLUSION

The risk prediction model was accurate and externally validated using high quality registry data. This is potentially suitable to be deployed as part of quality improvement analytics in stroke care to enable the fair comparison of stroke mortality outcomes across hospitals and health systems across countries.

Multilevel Quality Indicators: Methodology and Monte Carlo Evidence

BACKGROUND

Quality indicators are frequently used to assess the performance of health care providers, in particular hospitals. Established approaches to the design of such indicators are subject to distortions due to indirect standardization and high variance of estimators. Indicators for geographical regions are rarely considered.

OBJECTIVES

To develop and evaluate a methodology of multilevel quality indicators (MQIs) for both health care providers and geographical regions.

RESEARCH DESIGN

We formally derived MQIs from a statistical multilevel model, which may include characteristics of patients, providers, and regions. We used Monte Carlo simulation to assess the performance of MQIs relative to established approaches based on the standardized mortality/morbidity ratio (SMR) and the risk-standardized mortality rate (RSMR).

MEASURES

Rank correlation between true provider/region effects and quality indicator estimates; shares of the 10% best and 10% worst providers identified by the quality indicators.

RESULTS

The proposed MQIs are (1) standardized hospital outcome rate (SHOR), (2) regional SHOR, and (3) regional standardized patient outcome rate. Monte Carlo simulations indicated that the SHOR provides substantially better estimates of provider performance than the SMR and risk-standardized mortality rate in almost all scenarios. The regional standardized patient outcome rate was slightly more stable than the regional SMR. We also found that modeling of regional characteristics generally improves the adequacy of provider-level estimates.

CONCLUSIONS

MQIs methodology facilitates adequate and efficient estimation of quality indicators for both health care providers and geographical regions.