Development and validation of a model that uses enhanced administrative data to predict mortality in patients with sepsis

Machine learning for the prediction of sepsis-related death: a systematic review and meta-analysis

BACKGROUND AND OBJECTIVES

Sepsis is accompanied by a considerably high risk of mortality in the short term, despite the availability of recommended mortality risk assessment tools. However, these risk assessment tools seem to have limited predictive value. With the gradual integration of machine learning into clinical practice, some researchers have attempted to employ machine learning for early mortality risk prediction in sepsis patients. Nevertheless, there is a lack of comprehensive understanding regarding the construction of predictive variables using machine learning and the value of various machine learning methods. Thus, we carried out this systematic review and meta-analysis to explore the predictive value of machine learning for sepsis-related death at different time points.

METHODS

PubMed, Embase, Cochrane, and Web of Science databases were searched until August 9th, 2022. The risk of bias in predictive models was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). We also performed subgroup analysis according to time of death and type of model and summarized current predictive variables used to construct models for sepsis death prediction.

RESULTS

Fifty original studies were included, covering 104 models. The combined Concordance index (C-index), sensitivity, and specificity of machine learning models were 0.799, 0.81, and 0.80 in the training set, and 0.774, 0.71, and 0.68 in the validation set, respectively. Machine learning outperformed conventional clinical scoring tools and showed excellent C-index, sensitivity, and specificity in different subgroups. Random Forest (RF) and eXtreme Gradient Boosting (XGBoost) are the preferred machine learning models because they showed more favorable accuracy with similar modeling variables. This study found that lactate was the most frequent predictor but was seriously ignored by current clinical scoring tools.

CONCLUSION

Machine learning methods demonstrate relatively favorable accuracy in predicting the mortality risk in sepsis patients. Given the limitations in accuracy and applicability of existing prediction scoring systems, there is an opportunity to explore updates based on existing machine learning approaches. Specifically, it is essential to develop or update more suitable mortality risk assessment tools based on the specific contexts of use, such as emergency departments, general wards, and intensive care units.

Linking Sepsis with chronic arterial hypertension, diabetes mellitus, and socioeconomic factors in the United States: A scoping review

RATIONALE

Sepsis is a syndrome of life-threatening organ dysfunction caused by a dysregulated host immune response to infection. Social risk factors including location and poverty are associated with sepsis-related disparities. Understanding the social and biological phenotypes linked with the incidence of sepsis is warranted to identify the most at-risk populations. We aim to examine how factors in disadvantage influence health disparities related to sepsis.

METHODS

A scoping review was performed for English-language articles published in the United States from 1990 to 2022 on PubMed, Web of Science, and Scopus. Of the 2064 articles found, 139 met eligibility criteria and were included for review.

RESULTS

There is consistency across the literature of disproportionately higher rates of sepsis incidence, mortality, readmissions, and associated complications, in neighborhoods with socioeconomic disadvantage and significant poverty. Chronic arterial hypertension and diabetes mellitus also occur more frequently in the same geographic distribution as sepsis, suggesting a potential shared pathophysiology.

CONCLUSIONS

The distribution of chronic arterial hypertension, diabetes mellitus, social risk factors associated with socioeconomic disadvantage, and sepsis incidence, are clustered in specific geographical areas and linked by endothelial dysfunction. Such population factors can be utilized to create equitable interventions aimed at mitigating sepsis incidence and sepsis-related disparities.

Can markers of disease severity improve the predictive power of claims-based multimorbidity indices?

BACKGROUND

Claims-based measures of multimorbidity, which evaluate the presence of a defined list of diseases, are limited in their ability to predict future outcomes. We evaluated whether claims-based markers of disease severity could improve assessments of multimorbid burden.

METHODS

We developed 7 dichotomous markers of disease severity which could be applied to a range of diseases using claims data. These markers were based on the number of disease-associated outpatient visits, emergency department visits, and hospitalizations made by an individual over a defined interval; whether an individual with a given disease had outpatient visits to a specialist who typically treats that disease; and ICD-9 codes which connote more versus less advanced or symptomatic manifestations of a disease. Using Medicare claims linked with Health and Retirement Study data, we tested whether including these markers improved ability to predict ADL decline, IADL decline, hospitalization, and death compared to equivalent models which only included the presence or absence of diseases.

RESULTS

Of 5012 subjects, median age was 76 years and 58% were female. For a majority of diseases tested individually, adding each of the 7 severity markers yielded minimal increase in c-statistic (≤0.002) for outcomes of ADL decline and mortality compared to models considering only the presence versus absence of disease. Gains in predictive power were more substantial for a small number of individual diseases. Inclusion of the most promising marker in multi-disease multimorbidity indices yielded minimal gains in c-statistics (<0.001-0.007) for predicting ADL decline, IADL decline, hospitalization, and death compared to indices without these markers.

CONCLUSIONS

Claims-based markers of disease severity did not contribute meaningfully to the ability of multimorbidity indices to predict ADL decline, mortality, and other important outcomes.

Development and validation of risk-adjusted quality indicators for the long-term outcome of acute sepsis care in German hospitals based on health claims data

Background

Methods for assessing long-term outcome quality of acute care for sepsis are lacking. We investigated a method for measuring long-term outcome quality based on health claims data in Germany.

Materials and methods

Analyses were based on data of the largest German health insurer, covering 32% of the population. Cases (aged 15 years and older) with ICD-10-codes for severe sepsis or septic shock according to sepsis-1-definitions hospitalized in 2014 were included. Short-term outcome was assessed by 90-day mortality; long-term outcome was assessed by a composite endpoint defined by 1-year mortality or increased dependency on chronic care. Risk factors were identified by logistic regressions with backward selection. Hierarchical generalized linear models were used to correct for clustering of cases in hospitals. Predictive validity of the models was assessed by internal validation using bootstrap-sampling. Risk-standardized mortality rates (RSMR) were calculated with and without reliability adjustment and their univariate and bivariate distributions were described.

Results

Among 35,552 included patients, 53.2% died within 90 days after admission; 39.8% of 90-day survivors died within the first year or had an increased dependency on chronic care. Both risk-models showed a sufficient predictive validity regarding discrimination [AUC = 0.748 (95% CI: 0.742; 0.752) for 90-day mortality; AUC = 0.675 (95% CI: 0.665; 0.685) for the 1-year composite outcome, respectively], calibration (Brier Score of 0.203 and 0.220; calibration slope of 1.094 and 0.978), and explained variance (R ² = 0.242 and R ² = 0.111). Because of a small case-volume per hospital, applying reliability adjustment to the RSMR led to a great decrease in variability across hospitals [from median (1st quartile, 3rd quartile) 54.2% (44.3%, 65.5%) to 53.2% (50.7%, 55.9%) for 90-day mortality; from 39.2% (27.8%, 51.1%) to 39.9% (39.5%, 40.4%) for the 1-year composite endpoint]. There was no substantial correlation between the two endpoints at hospital level (observed rates: ρ = 0, p = 0.99; RSMR: ρ = 0.017, p = 0.56; reliability-adjusted RSMR: ρ = 0.067; p = 0.026).

Conclusion

Quality assurance and epidemiological surveillance of sepsis care should include indicators of long-term mortality and morbidity. Claims-based risk-adjustment models for quality indicators of acute sepsis care showed satisfactory predictive validity. To increase reliability of measurement, data sources should cover the full population and hospitals need to improve ICD-10-coding of sepsis.

An investigation of temperature and fever burdens in patients with sepsis admitted from the emergency department to the hospital

Aim

We sought to collect granular data on temperature burden to further explore existing conflicting information on the relationship between temperature alterations and outcomes in patients with sepsis requiring hospital admission.

Methods

This was a prospective cohort study that enrolled a convenience sample of patients with sepsis or septic shock admitted to the hospital from the emergency department (ED). A "unit of temperature burden (UTB)" was defined as >1°C (1.8°F) above or below 37°C (98.6°F) for 1 min. Fever burden was defined as the number of UTBs >38°C (100.4°F). The primary objective was to calculate the fever burden in patients with sepsis during their ED stay. This was analyzed for patients who present to triage febrile or hypothermic and also for those who developed temperature abnormalities during their ED stay. The secondary objectives were correlating fever and hypothermia burden with in-hospital mortality, Systemic Inflammatory Response Syndrome (SIRS) criteria, and the quick Sequential (Sepsis-Associated) Organ Failure Assessment (qSOFA) score and identification of patients who may benefit from early implementation of targeted temperature management.

Results

A total of 256 patients met the inclusion criteria. The mean age of patients was 60.1 ± 18.4 years; 46% were female and 29.6% were black. The median (interquartile range [IQR]) fever burden for the fever in triage cohort (n = 99) was 364.6 (174.3-716.8) UTB and for the no fever in triage cohort (n = 157) was 179.3 (80.9-374.0) UTB (p = 0.005). The two groups had similar in-hospital mortality (6.1 vs 8.3%; p = 0.5). The median fever burden for the fever anytime cohort was 303.8 (IQR 138.8-607.9) UTB and they had lower mortality than the no fever anytime cohort (4.7% vs 11.2%; p = 0.052). Patients with fever at triage had higher mean SIRS criteria than those without (2.8 vs 2.0; p < 0.001) while qSOFA points were similar (p = 0.199). A total of 27 patients had hypothermia during their ED stay and these patients were older with higher mean SIRS criteria.

Conclusions

Patients with sepsis and septic shock have a significant temperature burden in the ED. When comparing patients who had fever at any time during their ED stay with those who never had a fever, a trend toward an inverse relationship between fever burden and mortality was found.

Predicting Sepsis Mortality in a Population-Based National Database: Machine Learning Approach

BACKGROUND

Although machine learning (ML) algorithms have been applied to point-of-care sepsis prognostication, ML has not been used to predict sepsis mortality in an administrative database. Therefore, we examined the performance of common ML algorithms in predicting sepsis mortality in adult patients with sepsis and compared it with that of the conventional context knowledge-based logistic regression approach.

OBJECTIVE

The aim of this study is to examine the performance of common ML algorithms in predicting sepsis mortality in adult patients with sepsis and compare it with that of the conventional context knowledge-based logistic regression approach.

METHODS

We examined inpatient admissions for sepsis in the US National Inpatient Sample using hospitalizations in 2010-2013 as the training data set. We developed four ML models to predict in-hospital mortality: logistic regression with least absolute shrinkage and selection operator regularization, random forest, gradient-boosted decision tree, and deep neural network. To estimate their performance, we compared our models with the Super Learner model. Using hospitalizations in 2014 as the testing data set, we examined the models' area under the receiver operating characteristic curve (AUC), confusion matrix results, and net reclassification improvement.

RESULTS

Hospitalizations of 923,759 adults were included in the analysis. Compared with the reference logistic regression (AUC: 0.786, 95% CI 0.783-0.788), all ML models showed superior discriminative ability (P<.001), including logistic regression with least absolute shrinkage and selection operator regularization (AUC: 0.878, 95% CI 0.876-0.879), random forest (AUC: 0.878, 95% CI 0.877-0.880), xgboost (AUC: 0.888, 95% CI 0.886-0.889), and neural network (AUC: 0.893, 95% CI 0.891-0.895). All 4 ML models showed higher sensitivity, specificity, positive predictive value, and negative predictive value compared with the reference logistic regression model (P<.001). We obtained similar results from the Super Learner model (AUC: 0.883, 95% CI 0.881-0.885).

CONCLUSIONS

ML approaches can improve sensitivity, specificity, positive predictive value, negative predictive value, discrimination, and calibration in predicting in-hospital mortality in patients hospitalized with sepsis in the United States. These models need further validation and could be applied to develop more accurate models to compare risk-standardized mortality rates across hospitals and geographic regions, paving the way for research and policy initiatives studying disparities in sepsis care.

A scoping review of registry captured indicators for evaluating quality of critical care in ICU

BACKGROUND

Excess morbidity and mortality following critical illness is increasingly attributed to potentially avoidable complications occurring as a result of complex ICU management (Berenholtz et al., J Crit Care 17:1-2, 2002; De Vos et al., J Crit Care 22:267-74, 2007; Zimmerman J Crit Care 1:12-5, 2002). Routine measurement of quality indicators (QIs) through an Electronic Health Record (EHR) or registries are increasingly used to benchmark care and evaluate improvement interventions. However, existing indicators of quality for intensive care are derived almost exclusively from relatively narrow subsets of ICU patients from high-income healthcare systems. The aim of this scoping review is to systematically review the literature on QIs for evaluating critical care, identify QIs, map their definitions, evidence base, and describe the variances in measurement, and both the reported advantages and challenges of implementation.

METHOD

We searched MEDLINE, EMBASE, CINAHL, and the Cochrane libraries from the earliest available date through to January 2019. To increase the sensitivity of the search, grey literature and reference lists were reviewed. Minimum inclusion criteria were a description of one or more QIs designed to evaluate care for patients in ICU captured through a registry platform or EHR adapted for quality of care surveillance.

RESULTS

The search identified 4780 citations. Review of abstracts led to retrieval of 276 full-text articles, of which 123 articles were accepted. Fifty-one unique QIs in ICU were classified using the three components of health care quality proposed by the High Quality Health Systems (HQSS) framework. Adverse events including hospital acquired infections (13.7%), hospital processes (54.9%), and outcomes (31.4%) were the most common QIs identified. Patient reported outcome QIs accounted for less than 6%. Barriers to the implementation of QIs were described in 35.7% of articles and divided into operational barriers (51%) and acceptability barriers (49%).

CONCLUSIONS

Despite the complexity and risk associated with ICU care, there are only a small number of operational indicators used. Future selection of QIs would benefit from a stakeholder-driven approach, whereby the values of patients and communities and the priorities for actionable improvement as perceived by healthcare providers are prioritized and include greater focus on measuring discriminable processes of care.

A critical appraisal of the prognostic predictive models for patients with sepsis: Which model can be applied in clinical practice?

BACKGROUND

Sepsis is associated with high mortality and predictive models can help in clinical decision-making. The objective of this study was to carry out a systematic review of these models.

METHODS

In 2019, we conducted a systematic review in MEDLINE and EMBASE (CDR42018111121:PROSPERO) of articles that developed predictive models for mortality in septic patients (inclusion criteria). We followed the CHARMS recommendations (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies), extracting the information from its 11 domains (Source of data, Participants, etc). We determined the risk of bias and applicability (participants, outcome, predictors and analysis) through PROBAST (Prediction model Risk Of Bias ASsessment Tool).

RESULTS

A total of 14 studies were included. In the CHARMS extraction, the models found showed great variability in its 11 domains. Regarding the PROBAST checklist, only one article had an unclear risk of bias as it did not indicate how missing data were handled while the others all had a high risk of bias. This was mainly due to the statistical analysis (inadequate sample size, handling of continuous predictors, missing data and selection of predictors), since 13 studies had a high risk of bias. Applicability was satisfactory in six articles. Most of the models integrate predictors from routine clinical practice. Discrimination and calibration were assessed for almost all the models, with the area under the ROC curve ranging from 0.59 to 0.955 and no lack of calibration. Only three models were externally validated and their maximum discrimination values in the derivation were from 0.712 and 0.84. One of them (Osborn) had undergone multiple validation studies.

DISCUSSION

Despite most of the studies showing a high risk of bias, we very cautiously recommend applying the Osborn model, as this has been externally validated various times.

Short-Term Safety of Nicotine Replacement in Smokers Hospitalized With Coronary Heart Disease

Background

Little is known about the safety of nicotine replacement therapy (NRT) in smokers hospitalized with coronary heart disease.

Methods and Results

We examined the short‐term safety of NRT use among smokers hospitalized for coronary heart disease in a geographically and structurally diverse sample of US hospitals in the year 2014. We compared smokers who started NRT in the first 2 days of hospitalization with smokers without any exposure to NRT and adjusted for baseline differences through propensity score matching. Outcomes included inpatient mortality, hospital length of stay, and 1‐month readmission. From 270 hospitals, we included 27 459 smokers (mean age, 58 years; 69% men; 56.9% in intensive care unit), of whom 4885 (17.8%) received NRT (97.2% used the nicotine patch, at a median dose of 21 mg/d for 3 days). After propensity matching, covariates were well balanced within each patient group. Among patients with myocardial infarction, compared with patients who did not receive NRT, those who received NRT showed no difference in mortality (2.1% versus 2.3%; P=0.98), mean length of stay (4.4±3.5 versus 4.3±3.3 days; P=0.60), or 1‐month readmission (15.8% versus 14.6%; P=0.31). Results were similar for patients undergoing percutaneous coronary intervention or coronary artery bypass surgery.

Conclusions

Among smokers hospitalized for treatment of coronary heart disease, use of NRT was not associated with any differences in short‐term outcomes. Given the known beneficial effects of NRT in treating nicotine withdrawal, reducing cravings, and promoting smoking cessation after discharge, our findings suggest that NRT is a safe and reasonable treatment option.

A Long Short-Term Memory Ensemble Approach for Improving the Outcome Prediction in Intensive Care Unit

In intensive care unit (ICU), it is essential to predict the mortality of patients and mathematical models aid in improving the prognosis accuracy. Recently, recurrent neural network (RNN), especially long short-term memory (LSTM) network, showed advantages in sequential modeling and was promising for clinical prediction. However, ICU data are highly complex due to the diverse patterns of diseases; therefore, instead of single LSTM model, an ensemble algorithm of LSTM (eLSTM) is proposed, utilizing the superiority of the ensemble framework to handle the diversity of clinical data. The eLSTM algorithm was evaluated by the acknowledged database of ICU admissions Medical Information Mart for Intensive Care III (MIMIC-III). The investigation in total of 18415 cases shows that compared with clinical scoring systems SAPS II, SOFA, and APACHE II, random forests classification algorithm, and the single LSTM classifier, the eLSTM model achieved the superior performance with the largest value of area under the receiver operating characteristic curve (AUROC) of 0.8451 and the largest area under the precision-recall curve (AUPRC) of 0.4862. Furthermore, it offered an early prognosis of ICU patients. The results demonstrate that the eLSTM is capable of dynamically predicting the mortality of patients in complex clinical situations.

Risk Adjustment for Sepsis Mortality to Facilitate Hospital Comparisons Using Centers for Disease Control and Prevention's Adult Sepsis Event Criteria and Routine Electronic Clinical Data

Variability in hospital-level sepsis mortality rates may be due to differences in case mix, quality of care, or diagnosis and coding practices. Centers for Disease Control and Prevention's Adult Sepsis Event definition could facilitate objective comparisons of sepsis mortality rates between hospitals but requires rigorous risk-adjustment tools. We developed risk-adjustment models for Adult Sepsis Events using administrative and electronic health record data.

DESIGN

Retrospective cohort study.

SETTING

One hundred thirty-six U.S. hospitals in Cerner HealthFacts (derivation dataset) and 137 HCA Healthcare hospitals (validation dataset).

PATIENTS

A total of 95,154 hospitalized adult patients (derivation) and 201,997 patients (validation) meeting Centers for Disease Control and Prevention Adult Sepsis Event criteria.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We created logistic regression models of increasing complexity using administrative and electronic health record data to predict in-hospital mortality. An administrative model using demographics, comorbidities, and coded markers of severity of illness at admission achieved an area under the receiver operating curve of 0.776 (95% CI, 0.770-0.783) in the Cerner cohort, with diminishing calibration at higher baseline risk deciles. An electronic health record-based model that integrated administrative data with laboratory results, vasopressors, and mechanical ventilation achieved an area under the receiver operating curve of 0.826 (95% CI, 0.820-0.831) in the derivation cohort and 0.827 (95% CI, 0.824-0.829) in the validation cohort, with better calibration than the administrative model. Adding vital signs and Glasgow Coma Score minimally improved performance.

CONCLUSIONS

Models incorporating electronic health record data accurately predict hospital mortality for patients with Adult Sepsis Events and outperform models using administrative data alone. Utilizing laboratory test results, vasopressors, and mechanical ventilation without vital signs may achieve a good balance between data collection needs and model performance, but electronic health record-based models must be attentive to potential variability in data quality and availability. With ongoing testing and refinement of these risk-adjustment models, Adult Sepsis Event surveillance may enable more meaningful comparisons of hospital sepsis outcomes and provide an important window into quality of care.

Association Between Inpatient Echocardiography Use and Outcomes in Adult Patients With Acute Myocardial Infarction

IMPORTANCE

Guidelines recommend that patients with acute myocardial infarction (AMI) undergo echocardiography for assessment of cardiac structure and ejection fraction, but little is known about the association between echocardiography as used in routine clinical management of AMI and patient outcomes.

OBJECTIVE

To examine the association between risk-standardized hospital rates of transthoracic echocardiography and outcomes.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study of data from 397 US hospitals that contributed to the Premier Healthcare Informatics inpatient database from January 1, 2014, to December 31, 2014, used International Classification of Diseases, Ninth Revision (ICD-9) codes to identify 98 999 hospital admissions for patients with AMI. Data were analyzed between October 2017 and January 2019.

EXPOSURES

Rates of transthoracic echocardiography.

MAIN OUTCOMES AND MEASURES

Inpatient mortality, length of stay, total inpatient costs, and 3-month readmission rate.

RESULTS

Among the 397 hospitals with more than 25 admissions for AMI in 2014, a total of 98 999 hospital admissions for AMI were identified for analysis (38.2% women; mean [SD] age, 66.5 [13.6] years), of which 69 652 (70.4%) had at least 1 transthoracic echocardiogram performed. The median (IQR) hospital risk-standardized rate of echocardiography was 72.5% (62.6%-79.1%). In models that adjusted for hospital and patient characteristics, no difference was found in inpatient mortality (odds ratio [OR], 1.02; 95% CI, 0.88-1.19) or 3-month readmission (OR, 1.01; 95% CI, 0.93-1.10) between the highest and lowest quartiles of echocardiography use (median risk-standardized echocardiography use rates of 83% vs 54%, respectively). However, hospitals with the highest rates of echocardiography had modestly longer mean lengths of stay (0.23 days; 95% CI, 0.04-0.41; P = .01) and higher mean costs ($3164; 95% CI, $1843-$4485; P < .001) per admission compared with hospitals in the lowest quartile of use. Multiple sensitivity analyses yielded similar results.

CONCLUSIONS AND RELEVANCE

In patients with AMI, hospitals in the quartile with the highest rates of echocardiography showed greater hospital costs and length of stay but few differences in clinical outcomes compared with hospitals in the quartile with the lowest rates of echocardiography. These findings suggest that more selective use of echocardiography might be used without adversely affecting clinical outcomes, particularly in hospitals with high rates of echocardiography use.

TCR activation mimics CD127lowPD-1high phenotype and functional alterations of T lymphocytes from septic shock patients

Background

Sepsis is the leading cause of mortality for critically ill patients worldwide. Patients develop T lymphocyte dysfunctions leading to T-cell exhaustion associated with increased risk of death. As interleukin-7 (IL-7) is currently tested in clinical trials to reverse these dysfunctions, it is important to evaluate the expression of its specific CD127 receptor on the T-cell surface of patients with septic shock. Moreover, the CD127^lowPD-1^high phenotype has been proposed as a T-cell exhaustion marker in chronic viral infections but has never been evaluated in sepsis. The objective of this study was first to evaluate CD127 and CD127^lowPD-1^high phenotype in septic shock in parallel with functional T-cell alterations. Second, we aimed to reproduce septic shock–induced T-cell alterations in an ex vivo model.

Methods

CD127 expression was followed at the protein and mRNA levels in patients with septic shock and healthy volunteers. CD127^lowPD-1^high phenotype was also evaluated in parallel with T-cell functional alterations after ex vivo activation. To reproduce T-cell alterations observed in patients, purified T cells from healthy volunteers were activated ex vivo and their phenotype and function were evaluated.

Results

In patients, neither CD127 expression nor its corresponding mRNA transcript level was modified compared with normal values. However, the percentage of CD127^lowPD-1^high T cells was increased while T cells also presented functional alterations. CD127^lowPD-1^high T cells co-expressed HLA-DR, an activation marker, suggesting a role for T-cell activation in the development of this phenotype. Indeed, T-cell receptor (TCR) activation of normal T lymphocytes ex vivo reproduced the increase of CD127^lowPD-1^high T cells and functional alterations following a second stimulation, as observed in patients. Finally, in this model, as observed in patients, IL-7 could improve T-cell proliferation.

Conclusions

The proportion of CD127^lowPD-1^high T cells in patients was increased compared with healthy volunteers, although no global CD127 regulation was observed. Our results suggest that TCR activation participates in the occurrence of this T-cell population and in the development of T-cell alterations in septic shock. Furthermore, we provide an ex vivo model for the investigation of the pathophysiology of sepsis-induced T-cell immunosuppression and the testing of innovative immunostimulant treatments.

Electronic supplementary material

The online version of this article (10.1186/s13054-018-2305-5) contains supplementary material, which is available to authorized users.

Epidemiology of sepsis in Korea: a population-based study of incidence, mortality, cost and risk factors for death in sepsis

Objective

To investigate the epidemiology of sepsis in Korea and identify risk factors for death in sepsis.

Methods

We conducted a longitudinal, population-based epidemiological study of sepsis in Korea from 2005 to 2012 using the National Health Insurance Service-National Sample Cohort, a population-based cohort representing 2.2% of the Korean population. The primary objective was to assess the incidence, mortality and cost of sepsis. The secondary objective was to identify the risk factors for death in sepsis. Claim records of admitted adult patients (aged ≥15 years) were analyzed. Sepsis was defined as 1) bacterial or fungal infection or the conditions they often complicate, 2) prescription of intravenous antibiotics, and 3) presence of any organ dysfunction. Comorbidities were defined using the Charlson/Deyo method. Risk factors for 6-month mortality were assessed using multivariable logistic regression.

Results

A total of 22,882 cases were identified. Both incidence and 6-month mortality increased from 265.7 (95% confidence interval [CI], 254.7 to 277.1) to 453.1 (95% CI, 439.0 to 467.5) per 100,000 person-years (P-trend <0.001) and from 26.5% (95% CI, 24.4% to 28.8%) to 30.1% (95% CI, 28.4% to 31.9%), respectively. After standardization, the increasing trend of incidence was slower but still significant (P-trend <0.001), while that for mortality was not (P-trend 0.883). The average cost increased by 75.5% (P-trend <0.001). Multivariable logistic regression identified various risk factors for mortality.

Conclusion

The burden of sepsis in Korea was high and is expected to increase considering the aging population. Proactive measures to curtail this increase should be sought and implemented.

A machine learning-based model for 1-year mortality prediction in patients admitted to an Intensive Care Unit with a diagnosis of sepsis

INTRODUCTION

Sepsis is associated to a high mortality rate, and its severity must be evaluated quickly. The severity of illness scores used are intended to be applicable to all patient populations, and generally evaluate in-hospital mortality. However, patients with sepsis continue to be at risk of death after hospital discharge.

OBJECTIVE

To develop a model for predicting 1-year mortality in critical patients diagnosed with sepsis.

PATIENTS

The data corresponding to 5650 admissions of patients with sepsis from the Medical Information Mart for Intensive Care (MIMIC-III) database were evaluated, randomly divided as follows: 70% for training and 30% for validation.

DESIGN

A retrospective register-based cohort study was carried out. The clinical information of the first 24h after admission was used to develop a 1-year mortality prediction model based on Stochastic Gradient Boosting (SGB) methodology. Variable selection was addressed using Least Absolute Shrinkage and Selection Operator (LASSO) and SGB variable importance methodologies. The predictive power was evaluated using the area under the ROC curve (AUROC).

RESULTS

An AUROC of 0.8039 (95% confidence interval (CI): [0.8033 0.8045]) was obtained in the validation subset. The model exceeded the predictive performances obtained with traditional severity of disease scores in the same subset.

CONCLUSION

The use of assembly algorithms, such as SGB, for the generation of a customized model for sepsis yields more accurate 1-year mortality prediction than the traditional scoring systems such as SAPS II, SOFA or OASIS.

[Factors affecting in-hospital mortality in patients with sepsis: Development of a risk-adjusted model based on administrative data from German hospitals]

BACKGROUND

Inpatient administrative data from hospitals is already used nationally and internationally in many areas of internal and public quality assurance in healthcare. For sepsis as the principal condition, only a few published approaches are available for Germany. The aim of this investigation is to identify factors influencing hospital mortality by employing appropriate analytical methods in order to improve the internal quality management of sepsis.

METHODS

The analysis was based on data from 754,727 DRG cases of the CLINOTEL hospital network charged in 2015. The association then included 45 hospitals of all supply levels with the exception of university hospitals (range of beds: 100 to 1,172 per hospital). Cases of sepsis were identified via the ICD codes of their principal diagnosis. Multiple logistic regression analysis was used to determine the factors influencing in-hospital lethality for this population. The model was developed using sociodemographic and other potential variables that could be derived from the DRG data set, and taking into account current literature data. The model obtained was validated with inpatient administrative data of 2016 (51 hospitals, 850,776 DRG cases).

RESULTS

Following the definition of the inclusion criteria, 5,608 cases of sepsis (2016: 6,384 cases) were identified in 2015. A total of 12 significant and, over both years, stable factors were identified, including age, severity of sepsis, reason for hospital admission and various comorbidities. The AUC value of the model, as a measure of predictability, is above 0.8 (H-L test p>0.05, R² value=0.27), which is an excellent result.

CONCLUSION

The CLINOTEL model of risk adjustment for in-hospital lethality can be used to determine the mortality probability of patients with sepsis as principal diagnosis with a very high degree of accuracy, taking into account the case mix. Further studies are needed to confirm whether the model presented here will prove its value in the internal quality assurance of hospitals.

A risk-model for hospital mortality among patients with severe sepsis or septic shock based on German national administrative claims data

BACKGROUND

Sepsis is a major cause of preventable deaths in hospitals. Feasible and valid methods for comparing quality of sepsis care between hospitals are needed. The aim of this study was to develop a risk-adjustment model suitable for comparing sepsis-related mortality between German hospitals.

METHODS

We developed a risk-model using national German claims data. Since these data are available with a time-lag of 1.5 years only, the stability of the model across time was investigated. The model was derived from inpatient cases with severe sepsis or septic shock treated in 2013 using logistic regression with backward selection and generalized estimating equations to correct for clustering. It was validated among cases treated in 2015. Finally, the model development was repeated in 2015. To investigate secular changes, the risk-adjusted trajectory of mortality across the years 2010-2015 was analyzed.

RESULTS

The 2013 deviation sample consisted of 113,750 cases; the 2015 validation sample consisted of 134,851 cases. The model developed in 2013 showed good validity regarding discrimination (AUC = 0.74), calibration (observed mortality in 1st and 10th risk-decile: 11%-78%), and fit (R2 = 0.16). Validity remained stable when the model was applied to 2015 (AUC = 0.74, 1st and 10th risk-decile: 10%-77%, R2 = 0.17). There was no indication of overfitting of the model. The final model developed in year 2015 contained 40 risk-factors. Between 2010 and 2015 hospital mortality in sepsis decreased from 48% to 42%. Adjusted for risk-factors the trajectory of decrease was still significant.

CONCLUSIONS

The risk-model shows good predictive validity and stability across time. The model is suitable to be used as an external algorithm for comparing risk-adjusted sepsis mortality among German hospitals or regions based on administrative claims data, but secular changes need to be taken into account when interpreting risk-adjusted mortality.

Derivation and Validation of an In-Hospital Mortality Prediction Model Suitable for Profiling Hospital Performance in Heart Failure

BACKGROUND

Comparing heart failure (HF) outcomes across hospitals requires adequate risk adjustment. We aimed to develop and validate a model that can be used to compare quality of HF care across hospitals.

METHODS AND RESULTS

We included patients with HF aged ≥18 years admitted to one of 433 hospitals that participated in the Premier Inc Data Warehouse. This model (Premier) contained patient demographics, comorbidities, and acute conditions present on admission, derived from administrative and billing records. In a separate data set derived from electronic health records, we validated the Premier model by comparing hospital risk-standardized mortality rates calculated with the Premier model to those calculated with a validated clinical model containing laboratory data (LAPS [Laboratory-Based Acute Physiology Score]). Among the 200 832 admissions in the Premier Inc Data Warehouse, inpatient mortality was 4.0%. The model showed acceptable discrimination in the warehouse data (C statistic 0.75; 95% confidence interval, 0.74-0.76). In the validation data set, both the Premier model and the LAPS models showed acceptable discrimination (C statistic: Premier: 0.76 [95% confidence interval, 0.74-0.77]; LAPS: 0.78 [95% confidence interval, 0.76-0.80]). Risk-standardized mortality rates for both models ranged from 2% to 7%. A linear regression equation describing the association between Premier- and LAPS-specific mortality rates revealed a regression line with a slope of 0.71 (SE: 0.07). The correlation coefficient of the standardized mortality rates from the 2 models was 0.82.

CONCLUSIONS

Compared with a validated model derived from clinical data, an HF mortality model derived from administrative data showed highly correlated risk-standardized mortality rate estimates, suggesting it could be used to identify high- and low-performing hospitals for HF care.

Cardiac arrest risk standardization using administrative data compared to registry data

BACKGROUND

Methods for comparing hospitals regarding cardiac arrest (CA) outcomes, vital for improving resuscitation performance, rely on data collected by cardiac arrest registries. However, most CA patients are treated at hospitals that do not participate in such registries. This study aimed to determine whether CA risk standardization modeling based on administrative data could perform as well as that based on registry data.

METHODS AND RESULTS

Two risk standardization logistic regression models were developed using 2453 patients treated from 2000-2015 at three hospitals in an academic health system. Registry and administrative data were accessed for all patients. The outcome was death at hospital discharge. The registry model was considered the "gold standard" with which to compare the administrative model, using metrics including comparing areas under the curve, calibration curves, and Bland-Altman plots. The administrative risk standardization model had a c-statistic of 0.891 (95% CI: 0.876-0.905) compared to a registry c-statistic of 0.907 (95% CI: 0.895-0.919). When limited to only non-modifiable factors, the administrative model had a c-statistic of 0.818 (95% CI: 0.799-0.838) compared to a registry c-statistic of 0.810 (95% CI: 0.788-0.831). All models were well-calibrated. There was no significant difference between c-statistics of the models, providing evidence that valid risk standardization can be performed using administrative data.

CONCLUSIONS

Risk standardization using administrative data performs comparably to standardization using registry data. This methodology represents a new tool that can enable opportunities to compare hospital performance in specific hospital systems or across the entire US in terms of survival after CA.

A Comprehensive Multimorbidity Index for Predicting Mortality in Intensive Care Unit Patients

BACKGROUND

Accurate prediction of mortality for patients admitted to the intensive care units (ICUs) is an important component of medical care. However, little is known about the role of multimorbidity in predicting end of life for high-risk and vulnerable patients.

OBJECTIVE

The aim of the study was to derive and validate a multimorbidity risk model in an attempt to predict all-cause mortality at 6 and 12 months posthospital discharge.

METHODS

This is a retrospective, observational, clinical cohort study. Data were collected on 442,692 ICU patients who received care through the Veterans Administration between January 2003 and December 2013. The primary outcome was all-cause mortality at 6 and 12 months posthospital discharge. We divided the data into derivation (80%) and validation (20%) sets. Using multivariable logistic regression models, we compared prognostic models based on age, principal diagnosis groups, physiological markers, immunosuppressants, comorbidity categories, and a newly developed multimorbidity index (MMI) based on 5695 comorbidities. The cross-validated area under the receiver operating characteristic curve (AUC) was used to report the accuracy of predicting all-cause mortality at 6 and 12 months of hospital discharge.

RESULTS

The average age of patients was 68.87 years (standard deviation = 12.1), 95.9% were males, 44.9% were widowed, divorced, or separated. The relative order of accuracy in predicting mortality was the MMI (AUC = 0.84, CI = 0.83-0.84), VA Inpatient Evaluation Center index (AUC = 0.80, CI = 0.79-0.81), principal diagnosis groups (AUC = 0.74, CI = 0.73-0.74), comorbidities (AUC = 0.69, CI = 0.68-0.70), physiological markers (AUC = 0.65, CI = 0.64-0.65), age (AUC = 0.60, CI = 0.60-0.61),and immunosuppressant use (AUC = 0.59, CI = 0.58-0.59).

CONCLUSIONS

The MMI improved the accuracy of predicting short- and long-term all-cause mortality for ICU patients. Further prospective studies are needed to validate the index in different clinical settings and test generalizability of results in patients outside the VA system of care.

Practice Patterns and Outcomes Associated With Use of Anticoagulation Among Patients With Atrial Fibrillation During Sepsis

IMPORTANCE

Atrial fibrillation (AF) during sepsis is associated with an increased risk of ischemic stroke during hospitalization, but risks and benefits associated with anticoagulation for AF during sepsis are unclear.

OBJECTIVE

To determine clinician practice patterns and patient risk of stroke and bleeding associated with use of anticoagulation for AF during sepsis.

DESIGN, SETTING, AND PARTICIPANTS

A retrospective cohort study using enhanced administrative claims data from approximately 20% of patients hospitalized in the United States July 1, 2010, to June 30, 2013, examined patients with AF during sepsis who did not have additional indications for therapeutic anticoagulation. Propensity score and instrumental variable analyses were used to evaluate risks of in-hospital stroke and bleeding associated with anticoagulation during sepsis.

EXPOSURES

Parenteral anticoagulants administered in doses greater than those used for prophylaxis of venous thromboembolism.

MAIN OUTCOMES AND MEASURES

Ischemic stroke and clinically significant bleeding events during hospitalization.

RESULTS

Of 113 511 patients hospitalized with AF and sepsis, 38 582 were included in our primary analysis (18 976 men and 19 606 women; mean [SD] age, 74.9 [11.7] years). A total of 13 611 patients (35.3%) received parenteral anticoagulants, while 24 971 (64.7%) did not. Hospital utilization rates of parenteral anticoagulants for AF during sepsis varied (median, 33%; 25th-75th percentile, 25%-43%). CHA2DS2VASc scores (congestive heart failure, hypertension, age ≥75 years [doubled], type 1 or type 2 diabetes, stroke or transient ischemic attack or thromboembolism [doubled], vascular disease [prior myocardial infarction, peripheral artery disease, or aortic plaque], age 65-75 years, sex category [female]) poorly discriminated the risk of ischemic stroke during sepsis (C statistic, 0.526). Among 27 010 propensity score-matched patients, rates of in-hospital ischemic stroke events did not differ significantly between patients who did (174 of 13 505 [1.3%]) and did not (185 of 13 505 [1.4%]) receive parenteral anticoagulation (relative risk [RR], 0.94; 95% CI, 0.77-1.15). Clinically significant bleeding occurred more often among patients who received parenteral anticoagulation (1163 of 13 505 [8.6%]) than patients who did not receive parenteral anticoagulation (979 of 13 505 [7.2%]; RR, 1.21; 95% CI, 1.10-1.32). Risk of ischemic stroke associated with parenteral anticoagulation did not differ significantly between patients with preexisting (RR, 1.12; 95% CI, 0.86-1.44) or newly diagnosed AF (RR, 0.85; 95% CI 0.57-1.27; P = .31 for interaction). Results were robust to multiple sensitivity analyses, including hospital utilization rates of parenteral anticoagulation for AF as an instrument for anticoagulation exposure (RR for stroke, 1.08; 95% CI, 0.62-1.90; RR for bleeding, 1.23; 95% CI, 0.88-1.72).

CONCLUSIONS AND RELEVANCE

Among patients with AF during sepsis, parenteral anticoagulation was not associated with reduced risk of ischemic stroke and was associated with higher bleeding rates.

Sepsis surveillance from administrative data in the absence of a perfect verification

PURPOSE

Past studies of sepsis epidemiology did not address misclassification bias due to imperfect verification of sepsis detection methods to estimate the true prevalence.

METHODS

We examined 273,126 hospitalizations from 2008 to 2012 at a tertiary-care center to develop surveillance-aimed sepsis detection criteria, based on the presence of the sepsis-explicit International Classification of Diseases, Ninth Revision, Clinical Modification codes (995.92 or 785.52), blood culture orders, and antibiotics administration. We used Bayesian multinomial latent class models to estimate the true prevalence of sepsis, while adjusting for the imperfect sensitivity and specificity and the conditional dependence among the individual criteria.

RESULTS

The apparent annual prevalence of sepsis hospitalizations based on explicit International Classification of Diseases, Ninth Revision, Clinical Modification codes were 1.5%, 1.4%, 1.6%, 2.2%, and 2.5% for the years 2008 to 2012. Bayesian posterior estimates for the true prevalence of sepsis suggested that it remained stable from 2008, 19.2% (95% credible interval [CI]: 17.9%, 22.9%), to 2012, 17.8% (95% CI: 16.8%, 20.2%). The sensitivity of sepsis-explicit codes, however, increased from 7.6% (95% CI: 6.4%, 8.4%) in 2008 to 13.8% (95% CI: 12.2%, 14.9%) in 2012.

CONCLUSIONS

The true prevalence of sepsis remained high, but stable despite an increase in the sensitivity of sepsis-explicit codes in administrative data.

Estimating Ten-Year Trends in Septic Shock Incidence and Mortality in United States Academic Medical Centers Using Clinical Data

BACKGROUND

Reports that septic shock incidence is rising and mortality rates declining may be confounded by improving recognition of sepsis and changing coding practices. We compared trends in septic shock incidence and mortality in academic hospitals using clinical vs claims data.

METHODS

We identified all patients with concurrent blood cultures, antibiotics, and vasopressors for ≥ two consecutive days, and all patients with International Classification of Diseases, 9th edition (ICD-9) codes for septic shock, at 27 academic hospitals from 2005 to 2014. We compared annual incidence and mortality trends. We reviewed 967 records from three hospitals to estimate the accuracy of each method.

RESULTS

Of 6.5 million adult hospitalizations, 99,312 (1.5%) were flagged by clinical criteria, 82,350 (1.3%) by ICD-9 codes, and 44,651 (0.7%) by both. Sensitivity for clinical criteria was higher than claims (74.8% vs 48.3%; P < .01), whereas positive predictive value was comparable (83% vs 89%; P = .23). Septic shock incidence, based on clinical criteria, rose from 12.8 to 18.6 cases per 1,000 hospitalizations (average, 4.9% increase/y; 95% CI, 4.0%-5.9%), while mortality declined from 54.9% to 50.7% (average, 0.6% decline/y; 95% CI, 0.4%-0.8%). In contrast, septic shock incidence, based on ICD-9 codes, increased from 6.7 to 19.3 per 1,000 hospitalizations (19.8% increase/y; 95% CI, 16.6%-20.9%), while mortality decreased from 48.3% to 39.3% (1.2% decline/y; 95% CI, 0.9%-1.6%).

CONCLUSIONS

A clinical surveillance definition based on concurrent vasopressors, blood cultures, and antibiotics accurately identifies septic shock hospitalizations and suggests that the incidence of patients receiving treatment for septic shock has risen and mortality rates have fallen, but less dramatically than estimated on the basis of ICD-9 codes.

Real-Time Automated Sampling of Electronic Medical Records Predicts Hospital Mortality

BACKGROUND

Real-time automated continuous sampling of electronic medical record data may expeditiously identify patients at risk for death and enable prompt life-saving interventions. We hypothesized that a real-time electronic medical record-based alert could identify hospitalized patients at risk for mortality.

METHODS

An automated alert was developed and implemented to continuously sample electronic medical record data and trigger when at least 2 of 4 systemic inflammatory response syndrome criteria plus at least one of 14 acute organ dysfunction parameters was detected. The systemic inflammatory response syndrome and organ dysfunction alert was applied in real time to 312,214 patients in 24 hospitals and analyzed in 2 phases: training and validation datasets.

RESULTS

In the training phase, 29,317 (18.8%) triggered the alert and 5.2% of such patients died, whereas only 0.2% without the alert died (unadjusted odds ratio 30.1; 95% confidence interval, 26.1-34.5; P < .0001). In the validation phase, the sensitivity, specificity, area under the curve, and positive and negative likelihood ratios for predicting mortality were 0.86, 0.82, 0.84, 4.9, and 0.16, respectively. Multivariate Cox-proportional hazard regression model revealed greater hospital mortality when the alert was triggered (adjusted hazards ratio 4.0; 95% confidence interval, 3.3-4.9; P < .0001). Triggering the alert was associated with additional hospitalization days (+3.0 days) and ventilator days (+1.6 days; P < .0001).

CONCLUSION

An automated alert system that continuously samples electronic medical record data can be implemented, has excellent test characteristics, and can assist in the real-time identification of hospitalized patients at risk for death.

Volume-Mortality Relationships during Hospitalization with Severe Sepsis Exist Only at Low Case Volumes

RATIONALE

Volume-outcome associations have been demonstrated in conditions with high morbidity and mortality; however, the existing literature regarding such associations in sepsis is not definitive.

OBJECTIVES

To test the hypothesis that annual hospital severe sepsis case volume is associated with mortality during admissions with severe sepsis in teaching and nonteaching hospitals.

METHODS

This work was a retrospective cohort study of administrative data from the South Carolina State Inpatient Database using multivariate logistic regression and case mix adjustment.

MEASUREMENTS AND MAIN RESULTS

In the calendar year 2010, 9,815 patients were admitted with severe sepsis or septic shock. Hospitals were stratified into low- (0-75 cases/yr, n = 26), intermediate- (76-300 cases/yr, n = 19), and high (>300 cases/yr, n = 12) -volume tertiles. Patients admitted to hospitals with a low annual case volume for sepsis had higher adjusted odds of dying before discharge (odds ratio, 1.56; 95% confidence interval, 1.25-1.94) compared with patients admitted to high-volume hospitals. Hospitalization at intermediate-volume hospitals was not associated with a difference in mortality (odds ratio, 0.99; 95% confidence interval, 0.90-1.09) compared with high-volume hospitals. There was no difference between the mortality rates of intermediate- and high-volume hospitals at different severity of illness quartiles. Hospital length of stay differed significantly by hospital case volume (low = 8.0, intermediate = 12.7, high = 14.9 [d]; P < 0.0001).

CONCLUSIONS

Hospitals with low annual sepsis case volume are associated with higher mortality rates, whereas hospitals with intermediate sepsis case volumes are associated with similar mortality rates compared with hospitals with high case volumes.

A Severe Sepsis Mortality Prediction Model and Score for Use With Administrative Data

OBJECTIVE

Administrative data are used for research, quality improvement, and health policy in severe sepsis. However, there is not a sepsis-specific tool applicable to administrative data with which to adjust for illness severity. Our objective was to develop, internally validate, and externally validate a severe sepsis mortality prediction model and associated mortality prediction score.

DESIGN

Retrospective cohort study using 2012 administrative data from five U.S. states. Three cohorts of patients with severe sepsis were created: 1) International Classification of Diseases, 9th Revision, Clinical Modification codes for severe sepsis/septic shock, 2) Martin approach, and 3) Angus approach. The model was developed and internally validated in International Classification of Diseases, 9th Revision, Clinical Modification, cohort and externally validated in other cohorts. Integer point values for each predictor variable were generated to create a sepsis severity score.

SETTING

Acute care, nonfederal hospitals in New York, Maryland, Florida, Michigan, and Washington.

SUBJECTS

Patients in one of three severe sepsis cohorts: 1) explicitly coded (n = 108,448), 2) Martin cohort (n = 139,094), and 3) Angus cohort (n = 523,637) INTERVENTIONS: None.

MEASUREMENTS AND MAIN RESULTS

Maximum likelihood estimation logistic regression to develop a predictive model for in-hospital mortality. Model calibration and discrimination assessed via Hosmer-Lemeshow goodness-of-fit and C-statistics, respectively. Primary cohort subset into risk deciles and observed versus predicted mortality plotted. Goodness-of-fit demonstrated p value of more than 0.05 for each cohort demonstrating sound calibration. C-statistic ranged from low of 0.709 (sepsis severity score) to high of 0.838 (Angus cohort), suggesting good to excellent model discrimination. Comparison of observed versus expected mortality was robust although accuracy decreased in highest risk decile.

CONCLUSIONS

Our sepsis severity model and score is a tool that provides reliable risk adjustment for administrative data.

Outcomes of patients with healthcare-associated pneumonia: worse disease or sicker patients?

BACKGROUND

Healthcare-associated pneumonia (HCAP) is an entity distinct from community-acquired pneumonia (CAP). HCAP has a higher case-fatality rate, due either to HCAP organisms or to the health status of HCAP patients. The contribution of HCAP criteria to case-fatality rate is unknown.

METHODS

We conducted a retrospective review of adult patients admitted with a diagnosis of pneumonia from July 2007 through November 2011 to 491 US hospitals. HCAP was defined as having at least 1 of the following: prior hospitalization within 90 days, hemodialysis, admission from a skilled nursing facility, or immune suppression. We compared characteristics of patients with CAP and patients with HCAP and explored the contribution of HCAP criteria to case-fatality rate in a hierarchical generalized linear model.

RESULTS

Of 436,483 patients hospitalized with pneumonia, 149,963 (34.4%) had HCAP. Compared to CAP patients, HCAP patients were older, had more comorbidities, and were more likely to require intensive care unit (ICU) care. In-hospital case-fatality rate was higher among patients with HCAP, compared to those with CAP (11.1% vs 5.1%, P < .001). After adjustment for demographics, comorbidities, presence of other infections, early ICU admission, chronic and acute medications, early tests and therapies, and length of stay, HCAP remained associated with increased case-fatality rate (odds ratio [OR], 1.35 [95% confidence interval (CI), 1.32-1.39]); odds of death increased for each additional HCAP criterion (OR [95% CI]: 1 criterion, 1.27 [1.23-1.31], 2 criteria, 1.55 [1.49-1.62], and 3 or more criteria, 1.88 [1.72-2.06]).

CONCLUSIONS

After adjustment for differences in patient characteristics, HCAP was associated with greater case-fatality rate than CAP. This difference may be due to HCAP organisms or to HCAP criteria themselves.

Outcomes of Patients with Healthcare-Associated Pneumonia: Worse Disease or Sicker Patients?

Background.

Healthcare-associated pneumonia (HCAP) is an entity distinct from community-acquired pneumonia (CAP). HCAP has a higher case-fatality rate, due either to HCAP organisms or to the health status of HCAP patients. The contribution of HCAP criteria to case-fatality rate is unknown.

Methods.

We conducted a retrospective review of adult patients admitted with a diagnosis of pneumonia from July 2007 through November 2011 to 491 US hospitals. HCAP was defined as having at least 1 of the following: prior hospitalization within 90 days, hemodialysis, admission from a skilled nursing facility, or immune suppression. We compared characteristics of patients with CAP and patients with HCAP and explored the contribution of HCAP criteria to case-fatality rate in a hierarchical generalized linear model.

Results.

Of 436,483 patients hospitalized with pneumonia, 149,963 (34.4%) had HCAP. Compared to CAP patients, HCAP patients were older, had more comorbidities, and were more likely to require intensive care unit (ICU) care. In-hospital case-fatality rate was higher among patients with HCAP, compared to those with CAP (11.1% vs 5.1%, P < .001). After adjustment for demographics, comorbidities, presence of other infections, early ICU admission, chronic and acute medications, early tests and therapies, and length of stay, HCAP remained associated with increased case-fatality rate (odds ratio [OR], 1.35 [95% confidence interval (CI), 1.32-1.39]); odds of death increased for each additional HCAP criterion (OR [95% CI]: 1 criterion, 1.27 [1.23-1.31], 2 criteria, 1.55 [1.49-1.62], and 3 or more criteria, 1.88 [1.72-2.06]).

Conclusions.

After adjustment for differences in patient characteristics, HCAP was associated with greater case-fatality rate than CAP. This difference may be due to HCAP organisms or to HCAP criteria themselves.

Practice Patterns and Outcomes of Treatments for Atrial Fibrillation During Sepsis: A Propensity-Matched Cohort Study

BACKGROUND

Atrial fibrillation (AF) during sepsis is associated with increased morbidity and mortality, but practice patterns and outcomes associated with rate- and rhythm-targeted treatments for AF during sepsis are unclear.

METHODS

This was a retrospective cohort study using enhanced billing data from approximately 20% of United States hospitals. We identified factors associated with IV AF treatments (?-blockers [BBs], calcium channel blockers [CCBs], digoxin, or amiodarone) during sepsis. We used propensity score matching and instrumental variable approaches to compare mortality between AF treatments.

RESULTS

Among 39,693 patients with AF during sepsis, mean age was 77 ± 11 years, 49% were women, and 76% were white. CCBs were the most commonly selected initial AF treatment during sepsis (14,202 patients [36%]), followed by BBs (11,290 [28%]), digoxin (7,937 [20%]), and amiodarone (6,264 [16%]). Initial AF treatment selection differed according to geographic location, hospital teaching status, and physician specialty. In propensity-matched analyses, BBs were associated with lower hospital mortality when compared with CCBs (n = 18,720; relative risk [RR], 0.92; 95% CI, 0.86-0.97), digoxin (n = 13,994; RR, 0.79; 95% CI, 0.75-0.85), and amiodarone (n = 5,378; RR, 0.64; 95% CI, 0.61-0.69). Instrumental variable analysis showed similar results (adjusted RR fifth quintile vs first quintile of hospital BB use rate, 0.67; 95% CI, 0.58-0.79). Results were similar among subgroups with new-onset or preexisting AF, heart failure, vasopressor-dependent shock, or hypertension.

CONCLUSIONS

Although CCBs were the most frequently used IV medications for AF during sepsis, BBs were associated with superior clinical outcomes in all subgroups analyzed. Our findings provide rationale for clinical trials comparing the effectiveness of AF rate- and rhythm-targeted treatments during sepsis.

Identifying Pediatric Severe Sepsis and Septic Shock: Accuracy of Diagnosis Codes

OBJECTIVES

To evaluate accuracy of 2 established administrative methods of identifying children with sepsis using a medical record review reference standard.

STUDY DESIGN

Multicenter retrospective study at 6 US children's hospitals. Subjects were children >60 days to <19 years of age and identified in 4 groups based on International Classification of Diseases, Ninth Revision, Clinical Modification codes: (1) severe sepsis/septic shock (sepsis codes); (2) infection plus organ dysfunction (combination codes); (3) subjects without codes for infection, organ dysfunction, or severe sepsis; and (4) infection but not severe sepsis or organ dysfunction. Combination codes were allowed, but not required within the sepsis codes group. We determined the presence of reference standard severe sepsis according to consensus criteria. Logistic regression was performed to determine whether addition of codes for sepsis therapies improved case identification.

RESULTS

A total of 130 out of 432 subjects met reference SD of severe sepsis. Sepsis codes had sensitivity 73% (95% CI 70-86), specificity 92% (95% CI 87-95), and positive predictive value 79% (95% CI 70-86). Combination codes had sensitivity 15% (95% CI 9-22), specificity 71% (95% CI 65-76), and positive predictive value 18% (95% CI 11-27). Slight improvements in model characteristics were observed when codes for vasoactive medications and endotracheal intubation were added to sepsis codes (c-statistic 0.83 vs 0.87, P = .008).

CONCLUSIONS

Sepsis specific International Classification of Diseases, Ninth Revision, Clinical Modification codes identify pediatric patients with severe sepsis in administrative data more accurately than a combination of codes for infection plus organ dysfunction.

Unplanned Readmissions After Hospitalization for Severe Sepsis at Academic Medical Center-Affiliated Hospitals

OBJECTIVE

In the United States, national efforts to reduce hospital readmissions have been enacted, including the application of substantial insurance reimbursement penalties for hospitals with elevated rates. Readmissions after severe sepsis remain understudied and could possibly signify lapses in care and missed opportunities for intervention. We sought to characterize 7- and 30-day readmission rates following hospital admission for severe sepsis as well as institutional variations in readmission.

DESIGN

Retrospective analysis of 345,657 severe sepsis discharges from University HealthSystem Consortium hospitals in 2012.

SETTING

United States.

PATIENTS

We applied the commonly cited method described by Angus et al for identification of severe sepsis, including only discharges with sepsis present at admission.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We identified unplanned, all-cause readmissions within 7 and 30 days of discharge using claims-based algorithms. Using mixed-effects logistic regression, we determined factors associated with 30-day readmission. We used risk-standardized readmission rates to assess institutional variations. Among 216,328 eligible severe sepsis discharges, there were 14,932 readmissions within 7 days (6.9%; 95% CI, 6.8-7.0) and 43,092 within 30 days (19.9%; 95% CI, 19.8-20.1). Among those readmitted within 30 days, 66.9% had an infection and 40.3% had severe sepsis at readmission. Patient severity, length of stay, and specific diagnoses were associated with increased odds of 30-day readmission. Observed institutional 7-day readmission rates ranged from 0% to 12.3%, 30-day rates from 3.6% to 29.1%, and 30-day risk-standardized readmission rates from 14.1% to 31.1%. Greater institutional volume, teaching status, trauma services, location in the Northeast, and lower ICU rates were associated with poor risk-standardized readmission rate performance.

CONCLUSIONS

Severe sepsis readmission places a substantial burden on the healthcare system, with one in 15 and one in five severe sepsis discharges readmitted within 7 and 30 days, respectively. Hospitals and clinicians should be aware of this important sequela of severe sepsis.

Procedure-based severity index for inpatients: development and validation using administrative database

Background

Risk adjustment is important in studies using administrative databases. Although utilization of diagnostic and therapeutic procedures can represent patient severity, the usability of procedure records in risk adjustment is not well-documented. Therefore, we aimed to develop and validate a severity index calculable from procedure records.

Methods

Using the Japanese nationwide Diagnosis Procedure Combination database of acute-care hospitals, we identified patients discharged between 1 April 2012 and 31 March 2013 with an admission-precipitating diagnosis of acute myocardial infarction, congestive heart failure, acute cerebrovascular disease, gastrointestinal hemorrhage, pneumonia, or septicemia. Subjects were randomly assigned to the derivation cohort or the validation cohort. In the derivation cohort, we used multivariable logistic regression analysis to identify procedures performed on admission day which were significantly associated with in-hospital death, and a point corresponding to regression coefficient was assigned to each procedure. An index was then calculated in the validation cohort as sum of points for performed procedures, and performance of mortality-predicting model using the index and other patient characteristics was evaluated.

Results

Of the 539 385 hospitalizations included, 270 054 and 269 331 were assigned to the derivation and validation cohorts, respectively. Nineteen significant procedures were identified from the derivation cohort with points ranging from −3 to 23, producing a severity index with possible range of −13 to 69. In the validation cohort, c-statistic of mortality-predicting model was 0.767 (95 % confidence interval: 0.764–0.770). The ω-statistic representing contribution of the index relative to other variables was 1.09 (95 % confidence interval: 1.03–1.17).

Conclusions

Procedure-based severity index predicted mortality well, suggesting that procedure records in administrative database are useful for risk adjustment.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0889-x) contains supplementary material, which is available to authorized users.

The impact of hospital-onset Clostridium difficile infection on outcomes of hospitalized patients with sepsis

OBJECTIVE

To examine the impact of hospital-onset Clostridium difficile infection (HOCDI) on the outcomes of patients with sepsis.

BACKGROUND

Most prior studies that have addressed this issue lacked adequate matching to controls, suffered from small sample size, or failed to consider time to infection.

DESIGN

Retrospective cohort study.

SETTING AND PATIENTS

We identified adults with a principal or secondary diagnosis of sepsis who received care at 1 of the institutions that participated in a large multihospital database between July 1, 2004 and December 31, 2010. Among eligible patients with sepsis, we identified patients who developed HOCDI during their hospital stay.

MEASUREMENTS

We used propensity matching and date of diagnosis to match cases to patients without Clostridium difficile infections and compared outcomes between the 2 groups.

MAIN RESULTS

Of 218,915 sepsis patients, 2368 (1.08%) developed HOCDI. Unadjusted in-hospital mortality was significantly higher in HOCDI patients than controls (25% vs 10%, P < 0.001). After multivariate adjustment, in-hospital mortality rate was 24% in cases vs. 15% in controls. In an analysis limited to survivors, adjusted length of stay (LOS) among cases with Clostridium difficile infections was 5.1 days longer than controls (95% confidence interval: 4.4-5.8) and the median-adjusted cost increase was $4916 (P < 0.001).

CONCLUSIONS

After rigorous adjustment for time to diagnosis and presenting severity, hospital-acquired Clostridium difficile infection was associated with increased mortality, LOS, and cost. Our results can be used to assess the cost-effectiveness of prevention programs and suggest that efforts directed toward high-risk patient populations are needed.

Prevalence, treatment, and outcomes associated with OSA among patients hospitalized with pneumonia

BACKGROUND

OSA is associated with increased risks of respiratory complications following surgery. However, its relationship to the outcomes of hospitalized medical patients is unknown.

METHODS

We carried out a retrospective cohort study of patients with pneumonia at 347 US hospitals. We compared the characteristics, treatment, and risk of complications and mortality among patients with and without a diagnosis of OSA while adjusting for other patient and hospital factors.

RESULTS

Of the 250,907 patients studied, 15,569 (6.2%) had a diagnosis of OSA. Patients with OSA were younger (63 years vs 72 years), more likely to be men (53% vs 46%), more likely to be married (46% vs 38%), and had a higher prevalence of obesity (38% vs 6%), chronic pulmonary disease (68% vs 47%), and heart failure (28% vs 19%). Patients with OSA were more likely to receive invasive (18.1% vs 9.3%) and noninvasive (28.8% vs 6.8%) forms of ventilation upon hospital admission. After multivariable adjustment, OSA was associated with an increased risk of transfer to intensive care (OR, 1.54; 95% CI, 1.42-1.68) and intubation (OR, 1.68; 95% CI, 1.55-1.81) on or after the third hospital day, longer hospital stays (risk ratio [RR], 1.14; 95% CI, 1.13-1.15), and higher costs (RR, 1.22; 95% CI, 1.21-1.23) among survivors, but lower mortality (OR, 0.90; 95% CI, 0.84-0.98).

CONCLUSION

Among patients hospitalized for pneumonia, OSA is associated with higher initial rates of mechanical ventilation, increased risk of clinical deterioration, and higher resource use, yet a modestly lower risk of inpatient mortality.

Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis

BACKGROUND

Severe sepsis is a common and costly problem. Although consistently defined clinically by consensus conference since 1991, there have been several different implementations of the severe sepsis definition using ICD-9-CM codes for research. We conducted a single center, patient-level validation of 1 common implementation of the severe sepsis definition, the so-called "Angus" implementation.

METHODS

Administrative claims for all hospitalizations for patients initially admitted to general medical services from an academic medical center in 2009-2010 were reviewed. On the basis of ICD-9-CM codes, hospitalizations were sampled for review by 3 internal medicine-trained hospitalists. Chart reviews were conducted with a structured instrument, and the gold standard was the hospitalists' summary clinical judgment on whether the patient had severe sepsis.

RESULTS

Three thousand one hundred forty-six (13.5%) hospitalizations met ICD-9-CM criteria for severe sepsis by the Angus implementation (Angus-positive) and 20,142 (86.5%) were Angus-negative. Chart reviews were performed for 92 randomly selected Angus-positive and 19 randomly-selected Angus-negative hospitalizations. Reviewers had a κ of 0.70. The Angus implementation's positive predictive value was 70.7% [95% confidence interval (CI): 51.2%, 90.5%]. The negative predictive value was 91.5% (95% CI: 79.0%, 100%). The sensitivity was 50.4% (95% CI: 14.8%, 85.7%). Specificity was 96.3% (95% CI: 92.4%, 100%). Two alternative ICD-9-CM implementations had high positive predictive values but sensitivities of <20%.

CONCLUSIONS

The Angus implementation of the international consensus conference definition of severe sepsis offers a reasonable but imperfect approach to identifying patients with severe sepsis when compared with a gold standard of structured review of the medical chart by trained hospitalists.

Septic Shock Prediction for Patients with Missing Data

Sepsis and septic shock are common and potentially fatal conditions that often occur in intensive care unit (ICU) patients. Early prediction of patients at risk for septic shock is therefore crucial to minimizing the effects of these complications. Potential indications for septic shock risk span a wide range of measurements, including physiological data gathered at different temporal resolutions and gene expression levels, leading to a nontrivial prediction problem. Previous works on septic shock prediction have used small, carefully curated datasets or clinical measurements that may not be available for many ICU patients. The recent availability of a large, rich ICU dataset called MIMIC-II has provided the opportunity for more extensive modeling of this problem. However, such a large clinical dataset inevitably contains a substantial amount of missing data. We investigate how different imputation selection criteria and methods can overcome the missing data problem. Our results show that imputation methods in conjunction with predictive modeling can lead to accurate septic shock prediction, even if the features are restricted primarily to noninvasive measurements. Our models provide a generalized approach for predicting septic shock in any ICU patient.

Treatment with neuromuscular blocking agents and the risk of in-hospital mortality among mechanically ventilated patients with severe sepsis

OBJECTIVES

Recent trials suggest that treatment with neuromuscular blocking agents may improve survival in patients requiring mechanical ventilation for acute respiratory distress syndrome. We examined the association between receipt of a neuromuscular blocking agent and in-hospital mortality among mechanically ventilated patients with severe sepsis.

DESIGN

A pharmacoepidemiologic cohort study of patients with sepsis and a respiratory infection who had been admitted to intensive care and placed on mechanical ventilation within the first 2 days of hospitalization. We used propensity score matching and instrumental variable methods to compare the outcomes of patients treated with neuromuscular blocking agents within the first 2 hospital days to those who were not. Sensitivity analysis was used to model the effects of a hypothetical unmeasured confounder.

SETTING

Three hundred thirty-nine U.S. hospitals that participated in the Premier Perspective database between 2004 and 2006.

PATIENTS

Seven thousand eight hundred sixty-four patients met inclusion criteria, including 1,818 (23%) who were treated with a neuromuscular blocking agent by hospital day 2.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Patients who received neuromuscular blocking agents were younger (mean age, 62 vs 68), more likely to be treated with vasopressors (69% vs 65%) and had a lower in-hospital mortality rate (31.9% vs 38.3%, p < 0.001). In 3,518 patients matched on the propensity for treatment, receipt of a neuromuscular blocking agent was associated with a reduced risk of in-hospital mortality (risk ratio, 0.88; 95% CI, 0.80, 0.96). An analysis using the hospital neuromuscular blocking agent-prescribing rate as an instrumental variable found receipt of a neuromuscular blocking agent associated with a 4.3% (95% CI, -11.5%, 1.5%) reduction in in-hospital mortality.

CONCLUSIONS

Among mechanically ventilated patients with severe sepsis and respiratory infection, early treatment with a neuromuscular blocking agent is associated with lower in-hospital mortality.

Using highly detailed administrative data to predict pneumonia mortality

Background

Mortality prediction models generally require clinical data or are derived from information coded at discharge, limiting adjustment for presenting severity of illness in observational studies using administrative data.

Objectives

To develop and validate a mortality prediction model using administrative data available in the first 2 hospital days.

Research Design

After dividing the dataset into derivation and validation sets, we created a hierarchical generalized linear mortality model that included patient demographics, comorbidities, medications, therapies, and diagnostic tests administered in the first 2 hospital days. We then applied the model to the validation set.

Subjects

Patients aged ≥18 years admitted with pneumonia between July 2007 and June 2010 to 347 hospitals in Premier, Inc.’s Perspective database.

Measures

In hospital mortality.

Results

The derivation cohort included 200,870 patients and the validation cohort had 50,037. Mortality was 7.2%. In the multivariable model, 3 demographic factors, 25 comorbidities, 41 medications, 7 diagnostic tests, and 9 treatments were associated with mortality. Factors that were most strongly associated with mortality included receipt of vasopressors, non-invasive ventilation, and bicarbonate. The model had a c-statistic of 0.85 in both cohorts. In the validation cohort, deciles of predicted risk ranged from 0.3% to 34.3% with observed risk over the same deciles from 0.1% to 33.7%.

Conclusions

A mortality model based on detailed administrative data available in the first 2 hospital days had good discrimination and calibration. The model compares favorably to clinically based prediction models and may be useful in observational studies when clinical data are not available.

Contributors to variation in hospital spending for critically ill patients with sepsis

BACKGROUND

Costs of severe sepsis in the US exceeded $24 billion in 2007. Identifying the relative contributions of patient, hospital, and physician factors to the variation in hospital costs of sepsis could help target efforts to improve the value of care.

METHODS

We identified adults with a principal or secondary diagnosis of sepsis who received care between June 1, 2004 and June 30, 2006 at one of the hospitals participating in a multi-institutional database. We constructed a regression model to predict mean hospital costs that included patient characteristics, hospital mission and environment (e.g., teaching status, percentage of low-income patients), hospital fixed costs, and risk-adjusted length of stay, which encompasses hospital throughput, the incidence of complications, and other aspects of physician practice. To determine the contribution to cost variance by each predictor, we calculated the R(2).

RESULTS

At 189 hospitals, we identified 40,265 adults with sepsis who met inclusion criteria. The median cost of a hospitalization was $20,216. The model explained 69% of the hospital-level variation in the costs of hospitalization. Of explained variation, differences in patients' ages, comorbidities, and severity accounted for 20%; hospital mission and environment represented 16%; differences in hospital fixed costs, including acquisition costs and overhead, accounted for 19%; and wage index explained an additional 12%. Risk-adjusted length of stay comprised the final one-third of explained variation.

CONCLUSION

A large proportion of variation in the cost of caring for critically ill patients with sepsis across hospitals is related to differences in patient characteristics and immutable hospital characteristics, while nearly one-third is the result of differences in risk-adjusted length of stay.

IMPLICATIONS

Efforts to reduce spending on the critically ill should aim to understand determinants of practice style but should also focus on hospital throughput, overhead, acquisition, and labor costs.

Use of noninvasive ventilation in patients with acute respiratory failure, 2000-2009: a population-based study

RATIONALE

Although evidence supporting use of noninvasive ventilation (NIV) during acute exacerbations of chronic obstructive pulmonary disease (COPD) is strong, evidence varies widely for other causes of acute respiratory failure.

OBJECTIVES

To compare utilization trends and outcomes associated with NIV in patients with and without COPD.

METHODS

We identified 11,659,668 cases of acute respiratory failure from the Nationwide Inpatient Sample during years 2000 to 2009 and compared NIV utilization trends and failure rates for cases with or without a diagnosis of COPD.

MEASUREMENTS AND MAIN RESULTS

The proportion of patients with COPD who received NIV increased from 3.5% in 2000 to 12.3% in 2009 (250% increase), and the proportion of patients without COPD who received NIV increased from 1.2% in 2000 to 6.0% in 2009 (400% increase). The rate of increase in the use of NIV was significantly greater for patients without COPD (18.1% annual change) than for patients with COPD (14.3% annual change; P = 0.02). Patients without COPD were more likely to have failure of NIV requiring endotracheal intubation (adjusted odds ratio, 1.19; 95% confidence interval, 1.15-1.22; P < 0.0001). Patients in whom NIV failed had higher hospital mortality than patients receiving mechanical ventilation without a preceding trial of NIV (adjusted odds ratio, 1.14; 95% confidence interval, 1.11-1.17; P < 0.0001).

CONCLUSION

The use of NIV during acute respiratory failure has increased at a similar rate for all diagnoses, regardless of supporting evidence. However, NIV is more likely to fail in patients without COPD, and NIV failure is associated with increased mortality.

Incorporating initial treatments improves performance of a mortality prediction model for patients with sepsis

PURPOSE

Mortality prediction models can be used to adjust for presenting severity of illness in observational studies of treatment effectiveness. We aimed to determine the incremental benefit of adding information about critical care services to a sepsis mortality prediction model.

METHODS

In a retrospective cohort of 166 931 eligible sepsis patients at 309 hospitals, we developed nested logistic regression models to predict mortality at the patient level. Our initial model included only demographic information. We then added progressively more detailed information such as comorbidities and initial treatments. We calculated each model's area under the receiver operating characteristic curve (AUROC) and also used a sheaf coefficient analysis to determine the relative effect of each additional group of variables.

RESULTS

Model discrimination increased as more detailed patient information was added. With demographics alone, the AUROC was 0.59; adding comorbidities increased the AUROC to 0.67. The final model, which took into account mixed (hierarchical) effects at the hospital level as well as initial treatments administered within the first two hospital days, resulted in an AUROC of 0.78. The standardized sheaf coefficient for the initial treatments was approximately 30% greater than that for demographics or infection source.

CONCLUSIONS

A sepsis disease risk score that incorporates information about the use of mechanical ventilation and vasopressors is superior to models that rely only on demographic information and comorbidities. Until administrative datasets include clinical information (such as vital signs and laboratory results), models such as this one could allow researchers to conduct observational studies of treatment effectiveness in sepsis patients.

Update and review of racial disparities in sepsis

BACKGROUND

Sepsis is an excessive systemic inflammatory response activated by invasive infection. There has been substantial epidemiologic literature addressing perceived disparities in sepsis by demographic factors such as gender and race. There also have been multiple examinations of the disparities of sepsis with regard to environmental and socioeconomic factors. This paper reviews the current epidemiologic literature evaluating the association of race with the development of sepsis and its associated outcomes.

METHODS

Review of pertinent English-language literature.

RESULTS

Race is a marker of poverty, preexisting conditions, increased allostatic loads, and decreased access to health systems. Racial disparities and the incidence of sepsis likely are explained by a multiplicity of environmental factors that are not captured by administrative data.

CONCLUSION

Race is a surrogate for many intangible factors that lead to the development of sepsis and inferior outcomes.

Benchmarking strategies for measuring the quality of healthcare: problems and prospects

Over the last few years, increasing attention has been directed toward the problems inherent to measuring the quality of healthcare and implementing benchmarking strategies. Besides offering accreditation and certification processes, recent approaches measure the performance of healthcare institutions in order to evaluate their effectiveness, defined as the capacity to provide treatment that modifies and improves the patient's state of health. This paper, dealing with hospital effectiveness, focuses on research methods for effectiveness analyses within a strategy comparing different healthcare institutions. The paper, after having introduced readers to the principle debates on benchmarking strategies, which depend on the perspective and type of indicators used, focuses on the methodological problems related to performing consistent benchmarking analyses. Particularly, statistical methods suitable for controlling case-mix, analyzing aggregate data, rare events, and continuous outcomes measured with error are examined. Specific challenges of benchmarking strategies, such as the risk of risk adjustment (case-mix fallacy, underreporting, risk of comparing noncomparable hospitals), selection bias, and possible strategies for the development of consistent benchmarking analyses, are discussed. Finally, to demonstrate the feasibility of the illustrated benchmarking strategies, an application focused on determining regional benchmarks for patient satisfaction (using 2009 Lombardy Region Patient Satisfaction Questionnaire) is proposed.