The Apache III prognostic system: customized mortality predictions for Spanish ICU patients

Mortality, Functional Status, and Quality of Life after 5 Years of Patients Admitted to Critical Care for Spontaneous Intracerebral Hemorrhage

BACKGROUND

The objective of this study was to assess long-term outcome in patients with spontaneous intracerebral hemorrhage admitted to the intensive care unit.

METHODS

Mortality and Glasgow Outcome Scale, Barthel Index, and 5-level EQ-5D version (EQ-5D-5L) scores were analyzed in a multicenter cohort study of three Spanish hospitals (336 patients). Mortality was also analyzed in the Medical Information Mart for Intensive Care III (MIMIC-III) database.

RESULTS

The median (25th percentile-75th percentile) age was 62 (50-70) years, the median Glasgow Coma Score was 7 (4-11) points, and the median Acute Physiology and Chronic Health disease Classification System II (APACHE-II) score was 21 (15-26) points. Hospital mortality was 54.17%, mortality at 90 days was 56%, mortality at 1 year was 59.2%, and mortality at 5 years was 66.4%. In the Glasgow Outcome Scale, a normal or disabled self-sufficient situation was recorded in 21.5% of patients at 6 months, in 25.5% of patients after 1 year, and in 22.1% of patients after 5 years of follow-up (4.5% missing). The Barthel Index score of survivors improved over time: 50 (25-80) points at 6 months, 70 (35-95) points at 1 year, and 90 (40-100) points at 5 years (p < 0.001). Quality of life evaluated with the EQ-5D-5L at 1 year and 5 years indicated that greater than 50% of patients had no problems or slight problems in all items (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression). In the MIMIC-III study (N = 1354), hospital mortality was 31.83% and was 40.5% at 90 days and 56.2% after 5 years.

CONCLUSIONS

In patients admitted to the intensive care unit with a diagnosis of nontraumatic intracerebral hemorrhage, hospital mortality up to 90 days after admission is very high. Between 90 days and 5 years after admission, mortality is not high. A large percentage of survivors presented a significant deficit in quality of life and functional status, although with progressive improvement over time. Five years after the hemorrhagic stroke, a survival of 30% was observed, with a good functional status seen in 20% of patients who had been admitted to the hospital.

Predictive Modeling Using Artificial Intelligence and Machine Learning Algorithms on Electronic Health Record Data: Advantages and Challenges

The rapid adoption of electronic health record (EHR) systems in US hospitals from 2008 to 2014 produced novel data elements for analysis. Concurrent innovations in computing architecture and machine learning (ML) algorithms have made rapid consumption of health data feasible and a powerful engine for clinical innovation. In critical care research, the net convergence of these trends has resulted in an exponential increase in outcome prediction research. In the following article, we explore the history of outcome prediction in the intensive care unit (ICU), the growing use of EHR data, and the rise of artificial intelligence and ML (AI) in critical care.

Establishment of ICU Mortality Risk Prediction Models with Machine Learning Algorithm Using MIMIC-IV Database

Objective: The mortality rate of critically ill patients in ICUs is relatively high. In order to evaluate patients’ mortality risk, different scoring systems are used to help clinicians assess prognosis in ICUs, such as the Acute Physiology and Chronic Health Evaluation III (APACHE III) and the Logistic Organ Dysfunction Score (LODS). In this research, we aimed to establish and compare multiple machine learning models with physiology subscores of APACHE III—namely, the Acute Physiology Score III (APS III)—and LODS scoring systems in order to obtain better performance for ICU mortality prediction. Methods: A total number of 67,748 patients from the Medical Information Database for Intensive Care (MIMIC-IV) were enrolled, including 7055 deceased patients, and the same number of surviving patients were selected by the random downsampling technique, for a total of 14,110 patients included in the study. The enrolled patients were randomly divided into a training dataset (n = 9877) and a validation dataset (n = 4233). Fivefold cross-validation and grid search procedures were used to find and evaluate the best hyperparameters in different machine learning models. Taking the subscores of LODS and the physiology subscores that are part of the APACHE III scoring systems as input variables, four machine learning methods of XGBoost, logistic regression, support vector machine, and decision tree were used to establish ICU mortality prediction models, with AUCs as metrics. AUCs, specificity, sensitivity, positive predictive value, negative predictive value, and calibration curves were used to find the best model. Results: For the prediction of mortality risk in ICU patients, the AUC of the XGBoost model was 0.918 (95%CI, 0.915–0.922), and the AUCs of logistic regression, SVM, and decision tree were 0.872 (95%CI, 0.867–0.877), 0.872 (95%CI, 0.867–0.877), and 0.852 (95%CI, 0.847–0.857), respectively. The calibration curves of logistic regression and support vector machine performed better than the other two models in the ranges 0–40% and 70%–100%, respectively, while XGBoost performed better in the range of 40–70%. Conclusions: The mortality risk of ICU patients can be better predicted by the characteristics of the Acute Physiology Score III and the Logistic Organ Dysfunction Score with XGBoost in terms of ROC curve, sensitivity, and specificity. The XGBoost model could assist clinicians in judging in-hospital outcome of critically ill patients, especially in patients with a more uncertain survival outcome.

Learning to predict in-hospital mortality risk in the intensive care unit with attention-based temporal convolution network

BACKGROUND

Dynamic prediction of patient mortality risk in the ICU with time series data is limited due to high dimensionality, uncertainty in sampling intervals, and other issues. A new deep learning method, temporal convolution network (TCN), makes it possible to deal with complex clinical time series data in ICU. We aimed to develop and validate it to predict mortality risk using time series data from MIMIC III dataset.

METHODS

A total of 21,139 records of ICU stays were analysed and 17 physiological variables from the MIMIC III dataset were used to predict mortality risk. Then we compared the model performance of the attention-based TCN with that of traditional artificial intelligence (AI) methods.

RESULTS

The area under receiver operating characteristic (AUCROC) and area under precision-recall curve (AUC-PR) of attention-based TCN for predicting the mortality risk 48 h after ICU admission were 0.837 (0.824 -0.850) and 0.454, respectively. The sensitivity and specificity of attention-based TCN were 67.1% and 82.6%, respectively, compared to the traditional AI method, which had a low sensitivity (< 50%).

CONCLUSIONS

The attention-based TCN model achieved better performance in the prediction of mortality risk with time series data than traditional AI methods and conventional score-based models. The attention-based TCN mortality risk model has the potential for helping decision-making for critical patients.

TRIAL REGISTRATION

Data used for the prediction of mortality risk were extracted from the freely accessible MIMIC III dataset. The project was approved by the Institutional Review Boards of Beth Israel Deaconess Medical Center (Boston, MA) and the Massachusetts Institute of Technology (Cambridge, MA). Requirement for individual patient consent was waived because the project did not impact clinical care and all protected health information was deidentified. The data were accessed via a data use agreement between PhysioNet, a National Institutes of Health-supported data repository (https://www.physionet.org/), and one of us (Yu-wen Chen, Certification Number: 28341490). All methods were carried out in accordance with the institutional guidelines and regulations.

Utilizing heart rate variability to predict ICU patient outcome in traumatic brain injury

Background

Prediction of patient outcome in medical intensive care units (ICU) may help for development and investigation of early interventional strategies. Several ICU scoring systems have been developed and are used to predict clinical outcome of ICU patients. These scores are calculated from clinical physiological and biochemical characteristics of patients. Heart rate variability (HRV) is a correlate of cardiac autonomic regulation and has been evident as a marker of poor clinical prognosis. HRV can be measured from the electrocardiogram non-invasively and monitored in real time. HRV has been identified as a promising ‘electronic biomarker’ of disease severity. Traumatic brain injury (TBI) is a subset of critically ill patients admitted to ICU, with significant morbidity and mortality, and often difficult to predict outcomes. Changes of HRV for brain injured patients have been reported in several studies. This study aimed to utilize the continuous HRV collection from admission across the first 24 h in the ICU in severe TBI patients to develop a patient outcome prediction system.

Results

A feature extraction strategy was applied to measure the HRV fluctuation during time. A prediction model was developed based on HRV measures with a genetic algorithm for feature selection. The result (AUC: 0.77) was compared with earlier reported scoring systems (highest AUC: 0.76), encouraging further development and practical application.

Conclusions

The prediction models built with different feature sets indicated that HRV based parameters may help predict brain injury patient outcome better than the previously adopted illness severity scores.

Correlation between APACHE III score and sleep quality in ICU patients

Objective

To reveal the correlation between APACHE III score and sleep quality in patients in the intensive care unit (ICU).

Methods

This prospective, observational study included patients aged ≥18 years, who had been admitted to an integrated ICU for ≥48 h. Age, sex, Pittsburgh Sleep Quality Index (PSQI) prior to ICU, Numeric Rating Scales (NRS), noise, illumination, number of nursing interventions, Richards Campbell Sleep Questionnaire (RCSQ), and APACHE III score during sleep were evaluated.

Results

A total of 124 ICU patients were included, all with APACHE III scores < 60. APACHE III scores were not significantly associated with RCSQ scores. There were significant inverse associations between sleep quality in the ICU and PSQI score prior to ICU (odds ratio [OR] 0.587, 95% confidence interval [CI] 0.365, 0.945) and noise (OR 0.628, 95% CI 0.522, 0.756).

Conclusion

In ICU patients with APACHE III scores below 60 points, APACHE III score was not associated with sleep quality. PSQI score prior to ICU and noise were significantly inversely associated with sleep quality in this ICU patient population.

Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review

Background

Prognostic models—used in critical care medicine for mortality predictions, for benchmarking and for illness stratification in clinical trials—have been validated predominantly in high-income countries. These results may not be reproducible in low or middle-income countries (LMICs), not only because of different case-mix characteristics but also because of missing predictor variables. The study objective was to systematically review literature on the use of critical care prognostic models in LMICs and assess their ability to discriminate between survivors and non-survivors at hospital discharge of those admitted to intensive care units (ICUs), their calibration, their accuracy, and the manner in which missing values were handled.

Methods

The PubMed database was searched in March 2017 to identify research articles reporting the use and performance of prognostic models in the evaluation of mortality in ICUs in LMICs. Studies carried out in ICUs in high-income countries or paediatric ICUs and studies that evaluated disease-specific scoring systems, were limited to a specific disease or single prognostic factor, were published only as abstracts, editorials, letters and systematic and narrative reviews or were not in English were excluded.

Results

Of the 2233 studies retrieved, 473 were searched and 50 articles reporting 119 models were included. Five articles described the development and evaluation of new models, whereas 114 articles externally validated Acute Physiology and Chronic Health Evaluation, the Simplified Acute Physiology Score and Mortality Probability Models or versions thereof. Missing values were only described in 34% of studies; exclusion and or imputation by normal values were used. Discrimination, calibration and accuracy were reported in 94.0%, 72.4% and 25% respectively. Good discrimination and calibration were reported in 88.9% and 58.3% respectively. However, only 10 evaluations that reported excellent discrimination also reported good calibration. Generalisability of the findings was limited by variability of inclusion and exclusion criteria, unavailability of post-ICU outcomes and missing value handling.

Conclusions

Robust interpretations regarding the applicability of prognostic models are currently hampered by poor adherence to reporting guidelines, especially when reporting missing value handling. Performance of mortality risk prediction models in LMIC ICUs is at best moderate, especially with limitations in calibration. This necessitates continued efforts to develop and validate LMIC models with readily available prognostic variables, perhaps aided by medical registries.

Electronic supplementary material

The online version of this article (doi:10.1186/s13054-017-1930-8) contains supplementary material, which is available to authorized users.

Relationship between functional status prior to onset of critical illness and mortality: a prospective multicentre cohort study

This prospective study aimed to assess the association between prior functional status and hospital mortality for patients admitted to four intensive care units in Spain between 2006 and 2012. Prior functional status was classified into three groups, using a modification of the Glasgow Outcome Scale (GOS), including group 1 with no limitations on activities of daily living; group 2 with some limitations but self-sufficient; and group 3 who were dependent on others for their activities of daily living. Of the 1,757 patients considered (mean Simplified Acute Physiology Score [SAPS] predicted mortality 14.8% and hospital mortality 13.7%), group 1 had the lowest observed hospital mortality (8.3%) compared to the SAPS 3 predicted mortality (11.6%). The observed mortality for group 2 (20.6%) and group 3 (27.4%) were both higher than predicted (19.2% and 21.2% respectively; odds ratio [OR] 1.97, 95% confidence interval [CI] 1.38-2.82 for group 2 and OR 2.90, 95% CI 1.78-4.72 for group 3 compared to group 1). Combining prior functional status and Sequential Organ Failure Assessment (SOFA) score with SAPS 3 further improved the ability of the SAPS 3 scores in predicting hospital mortality (area under the receiver operating characteristic curve 0.85 [95% CI 0.82-0.88] versus 0.84 [95% CI 0.81-0.87] respectively). In summary, patients with limited functional status prior to ICU admission had a higher risk of observed hospital mortality than predicted. Assessing prior functional status using a relatively simple questionnaire, such as a modified GOS, has the potential to improve the accuracy of existing prognostic models.

Applicability of the APACHE II model to a lower middle income country

PURPOSE

To determine the utility of APACHE II in a low-and middle-income (LMIC) setting and the implications of missing data.

MATERIALS AND METHODS

Patients meeting APACHE II inclusion criteria admitted to 18 ICUs in Sri Lanka over three consecutive months had data necessary for the calculation of APACHE II, probabilities prospectively extracted from case notes. APACHE II physiology score (APS), probabilities, Standardised (ICU) Mortality Ratio (SMR), discrimination (AUROC), and calibration (C-statistic) were calculated, both by imputing missing measurements with normal values and by Multiple Imputation using Chained Equations (MICE).

RESULTS

From a total of 995 patients admitted during the study period, 736 had APACHE II probabilities calculated. Data availability for APS calculation ranged from 70.6% to 88.4% for bedside observations and 18.7% to 63.4% for invasive measurements. SMR (95% CI) was 1.27 (1.17, 1.40) and 0.46 (0.44, 0.49), AUROC (95% CI) was 0.70 (0.65, 0.76) and 0.74 (0.68, 0.80), and C-statistic was 68.8 and 156.6 for normal value imputation and MICE, respectively.

CONCLUSIONS

An incomplete dataset confounds interpretation of prognostic model performance in LMICs, wherein imputation using normal values is not a suitable strategy. Improving data availability, researching imputation methods and developing setting-adapted and simpler prognostic models are warranted.

Patients Admitted to Three Spanish Intensive Care Units for Poisoning: Type of Poisoning, Mortality, and Functioning of Prognostic Scores Commonly Used

Objectives. To evaluate the gravity and mortality of those patients admitted to the intensive care unit for poisoning. Also, the applicability and predicted capacity of prognostic scales most frequently used in ICU must be evaluated. Methods. Multicentre study between 2008 and 2013 on all patients admitted for poisoning. Results. The results are from 119 patients. The causes of poisoning were medication, 92 patients (77.3%), caustics, 11 (9.2%), and alcohol, 20 (16,8%). 78.3% attempted suicides. Mean age was 44.42 ± 13.85 years. 72.5% had a Glasgow Coma Scale (GCS) ≤8 points. The ICU mortality was 5.9% and the hospital mortality was 6.7%. The mortality from caustic poisoning was 54.5%, and it was 1.9% for noncaustic poisoning (p < 0.001). After adjusting for SAPS-3 (OR: 1.19 (1.02–1.39)) the mortality of patients who had ingested caustics was far higher than the rest (OR: 560.34 (11.64–26973.83)). There was considerable discrepancy between mortality predicted by SAPS-3 (26.8%) and observed (6.7%) (Hosmer-Lemeshow test: H = 35.10; p < 0.001). The APACHE-II (7,57%) and APACHE-III (8,15%) were no discrepancies. Conclusions. Admission to ICU for poisoning is rare in our country. Medication is the most frequent cause, but mortality of caustic poisoning is higher. APACHE-II and APACHE-III provide adequate predictions about mortality, while SAPS-3 tends to overestimate.

Predicting Mortality in Low-Income Country ICUs: The Rwanda Mortality Probability Model (R-MPM)

Introduction

Intensive Care Unit (ICU) risk prediction models are used to compare outcomes for quality improvement initiatives, benchmarking, and research. While such models provide robust tools in high-income countries, an ICU risk prediction model has not been validated in a low-income country where ICU population characteristics are different from those in high-income countries, and where laboratory-based patient data are often unavailable. We sought to validate the Mortality Probability Admission Model, version III (MPM₀-III) in two public ICUs in Rwanda and to develop a new Rwanda Mortality Probability Model (R-MPM) for use in low-income countries.

Methods

We prospectively collected data on all adult patients admitted to Rwanda’s two public ICUs between August 19, 2013 and October 6, 2014. We described demographic and presenting characteristics and outcomes. We assessed the discrimination and calibration of the MPM₀-III model. Using stepwise selection, we developed a new logistic model for risk prediction, the R-MPM, and used bootstrapping techniques to test for optimism in the model.

Results

Among 427 consecutive adults, the median age was 34 (IQR 25–47) years and mortality was 48.7%. Mechanical ventilation was initiated for 85.3%, and 41.9% received vasopressors. The MPM₀-III predicted mortality with area under the receiver operating characteristic curve of 0.72 and Hosmer-Lemeshow chi-square statistic p = 0.024. We developed a new model using five variables: age, suspected or confirmed infection within 24 hours of ICU admission, hypotension or shock as a reason for ICU admission, Glasgow Coma Scale score at ICU admission, and heart rate at ICU admission. Using these five variables, the R-MPM predicted outcomes with area under the ROC curve of 0.81 with 95% confidence interval of (0.77, 0.86), and Hosmer-Lemeshow chi-square statistic p = 0.154.

Conclusions

The MPM₀-III has modest ability to predict mortality in a population of Rwandan ICU patients. The R-MPM is an alternative risk prediction model with fewer variables and better predictive power. If validated in other critically ill patients in a broad range of settings, the model has the potential to improve the reliability of comparisons used for critical care research and quality improvement initiatives in low-income countries.

Evaluation of prognostic scale Thrombolysis In Myocardial Infarction and Killip. An ST-elevation myocardial infarction new scale

BACKGROUND

Prognostic systems are complex. So it is necessary to find tools, which are easy to use and have good calibration and discrimination.

OBJECTIVES

The objective of this study is to evaluate the usefulness of Killip, Thrombolysis In Myocardial Infarction (TIMI), and age to develop a new prognostic scale for patients with ST-elevation myocardial infarction (STEMI).

METHODS

The study population included all patients with STEMI consecutively admitted to the Intensive Care Unit of Carlos Haya Hospital, Malaga, Spain. Top variables included are Killip and TIMI, hospital mortality, intensive care unit stay, treatment received, and care times intervals.

RESULTS

The results are 806 patients; 75.6% men; age 63.11 ± 12.83 years old; TIMI, 3.57 ± 2.38; Killip I, 81.4%; and hospital mortality, 11.3%. Mortality increased in relation to age, TIMI, and Killip (P < .001). Receiver operating characteristic (ROC) area for TIMI is 0.832 (0.786-0.878) and Killip, 0.757 (0.698-0.822). Thrombolysis In Myocardial Infarction classification was associated with Killip and age by multiple linear regression. Patients were stratified into 5 groups according to Killip and age: Killip I and younger than 65 years (n = 369; mortality, 1.4%; odds ratio [OR], 1), Killip I and 65 to 75 years old (n = 173; mortality, 6.9%; OR, 5.43 [1.88-15.66]), Killip I and older than 75 years (n = 112; mortality, 18.9%; OR, 13.03 [4.69-36.21]), Killip II to III (n = 129; mortality, 31%; OR, 22.72 [12.55-85.29]), Killip IV (n = 20; mortality, 80%; OR, 291.2 [71.32-1189]). ROC area is 0.84 (0.798-0.883). We created a scale with scores based on the β coefficient of logistical regression.

CONCLUSIONS

The TIMI scale discriminated hospital mortality correctly for STEMI. It performed better than Killip alone and similar to a simple model that included age and Killip. The 2-variable model consists of a simple scale with 5 categories.

A new severity of illness scale using a subset of Acute Physiology And Chronic Health Evaluation data elements shows comparable predictive accuracy

OBJECTIVES

Severity of illness scores have gained considerable interest for their use in predicting outcomes such as mortality and length of stay. The most sophisticated scoring systems require the collection of numerous physiologic measurements, making their use in real-time difficult. A severity of illness score based on a few parameters that can be captured electronically would be of great benefit. Using a machine-learning technique known as particle swarm optimization, we attempted to reduce the number of physiologic parameters collected in the Acute Physiology, Age, and Chronic Health Evaluation IV system without losing predictive accuracy.

DESIGN

Retrospective cohort study of ICU admissions from 2007 to 2011.

SETTING

Eighty-six ICUs at 49 U.S. hospitals where an Acute Physiology, Age, and Chronic Health Evaluation IV system had been installed.

PATIENTS

81,087 admissions, of which 72,474 did not have any missing values.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Machine-learning algorithms were used to come up with the minimal set of variables that were capable of yielding an accurate severity of illness score: the Oxford Acute Severity of Illness Score. Predictive models of ICU mortality using Oxford Acute Severity of Illness Score were developed on admissions during 2007-2009 and validated on admissions during 2010-2011. The most parsimonious Oxford Acute Severity of Illness Score consisted of seven physiologic measurements, elective surgery, age, and prior length of stay. Predictive models of ICU mortality using Oxford Acute Severity of Illness Score achieved an area under the receiver operating characteristic curve of 0.88 and calibrated well.

CONCLUSIONS

A reduced severity of illness score had discrimination and calibration equivalent to more complex existing models. This was accomplished in large part using machine-learning algorithms, which can effectively account for the nonlinear associations between physiologic parameters and outcome.

Impaired postoperative hyperglycemic stress response associated with increased mortality in patients in the cardiothoracic surgery intensive care unit

OBJECTIVE

To describe the association of tight glycemic control with intensive insulin therapy and clinical outcome among patients in the cardiothoracic surgery intensive care unit.

METHODS

All patients who underwent cardiothoracic surgery and were admitted to the cardiothoracic surgery intensive care unit between September 13, 2007, and November 1, 2007, were enrolled. Clinical and metabolic data were prospectively collected. All patients received intensive insulin therapy using a nurse-driven dynamic protocol targeting blood glucose values of 80 to 110 mg/dL. Four stages of critical illness were defined as follows: acute critical illness (intensive care unit days 0-2), prolonged acute critical illness (intensive care unit 3 or more days), chronic critical illness (tracheotomy performed), and recovery (liberated from ventilator).

RESULTS

One hundred fourteen patients were enrolled. Seventy-three (64%) recovered during acute critical illness, 26 (23%) recovered during prolonged acute critical illness, and 15 (13%) progressed to chronic critical illness. All 6 deaths were among patients in chronic critical illness. Admission blood glucose and average blood glucose values for the first 12 hours were lower in patients who developed chronic critical illness and died and were higher in patients who developed chronic critical illness and survived (P = .007 and P = .007, respectively). Severe hypoglycemia (blood glucose <40 mg/dL) occurred once (0.03% of all measurements). Lower initial blood glucose values, which reflect an impaired stress response immediately after surgery, were associated with increased mortality, and a significant delay in achieving tight glycemic control with intensive insulin therapy was associated with prolonged intensive care unit course, but no increase in mortality.

CONCLUSION

The study findings suggest that acute postoperative hyperglycemia and its prompt correction with intensive insulin therapy are associated with favorable outcomes in patients in the cardiothoracic surgery intensive care unit.

Severity of illness scoring systems in the intensive care unit

OBJECTIVE

Adult intensive care unit prognostic models have been used for predicting patient outcome for three decades. The goal of this review is to describe the different versions of the main adult intensive care unit prognostic models and discuss their potential roles.

DATA SOURCE

PubMed search and review of the relevant medical literature.

SUMMARY

The main prognostic models for assessing the overall severity of illness in critically ill adults are Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and Mortality Probability Model. Simplified Acute Physiology Score and Mortality Probability Model have been updated to their third versions and Acute Physiology and Chronic Health Evaluation to its fourth version. The development of prognostic models is usually followed by internal and external validation and performance assessment. Performance is assessed by area under the receiver operating characteristic curve for discrimination and Hosmer-Lemeshow statistic for calibration. The areas under the receiver operating characteristic curve of Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III were 0.85, 0.88, and 0.82, respectively, and all these three fourth-generation models had good calibration. The models have been extensively used for case-mix adjustment in clinical research and epidemiology, but their role in benchmarking, performance improvement, resource use, and clinical decision support has been less well studied.

CONCLUSIONS

The fourth-generation Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score 3, Acute Physiology and Chronic Health Evaluation IV, and Mortality Probability Model0 III adult prognostic models, perform well in predicting mortality. Future studies are needed to determine their roles for benchmarking, performance improvement, resource use, and clinical decision support.

APACHE-II score and Killip class for patients with acute myocardial infarction

OBJECTIVE

To analyse the influence on the prognosis of intensive care unit (ICU) patients with acute myocardial infarction (AMI): prognostic index score, Killip class, AMI site, thrombolysis and other variables that might improve prognostic capacity and functioning of the APACHE-II index.

DESIGN

Cohort study using prospectively gathered ARIAM project data.

SETTING

ICUs from 129 Spanish hospitals.

PATIENTS

ICU-admitted AMI patients in ARIAM database during 4-year period were retrospectively studied.

MEASUREMENTS AND MAIN RESULTS

The sample comprised 6,458 patients, 76.8% males, age 64.97 +/- 12.56 years, APACHE-II score 9.49 +/- 7.03 points and ICU mortality 8.9%. Mortality was higher for females (p < 0.001), anterior AMI site (p < 0.001), previous AMI (p < 0.001), delay-to-hospital arrival >180 min (p = 0.003) and non-receipt of thrombolysis (p = 0.015). ICU mortality was related to age (p < 0.001) and APACHE-II score (p < 0.001). In multivariate analysis, it was related to APACHE-II (OR 1.16), age (OR 1.05), gender (OR 1.64), previous AMI (OR 1.57), anterior AMI (OR 2.05) and delay >180 min (OR 1.37). Killip class, gathered in 1,893 patients, was significantly associated with ICU mortality, and two predictive models were constructed for this group using multivariate analysis. Area under ROC curve was 0.94 in one (Killip class, age, gender, APACHE-II) versus 0.92 in the other (same variables without APACHE-II).

CONCLUSIONS

APACHE-II score and Killip class are useful for assessing the severity of patients with AMI and are complementary. Each can be used with a few commonly gathered clinical variables to construct prognostic models to assess severity. Their joint application yields a model with excellent discrimination capacity.

Comparison of noninvasive markers of fibrosis in patients with nonalcoholic fatty liver disease

BACKGROUND & AIMS

There is a need for a reliable and inexpensive noninvasive marker of hepatic fibrosis in patients with nonalcoholic fatty liver disease (NAFLD). We compared the performance of the FIB4 index (based on age, aspartate aminotransferase [AST] and alanine aminotransferase [ALT] levels, and platelet counts) with 6 other non-invasive markers of fibrosis in patients with NAFLD.

METHODS

Using a nation-wide database of 541 adults with NAFLD, jackknife-validated areas under receiver operating characteristic curves (AUROC) of FIB4 and 7 other markers were compared. The sensitivity at 90% specificity, 80% positive predictive value, and 90% negative predictive values were determined along with cutoffs for advanced fibrosis.

RESULTS

The median FIB4 score was 1.11 (interquartile range = 0.74-1.67). The jackknife-validated AUROC for FIB4 was 0.802 (95% confidence interval [CI], 0.758-0.847), which was higher than that of the NAFLD fibrosis score (0.768; 95% CI, 0.720-0.816; P = .09), Goteburg University Cirrhosis Index (0.743; 95% CI, 0.695-0.791; P < .01), AST:ALT ratio (0.742; 95% CI, 0.690-0.794; P < .015), AST:platelet ratio index (0.730; 95% CI, 0.681-0.779; P < .001), AST:platelet ratio (0.720; 95% CI, 0.669-0.770; P < .001), body mass index, AST:ALT, diabetes (BARD) score (0.70; P < .001), or cirrhosis discriminant score (0.666; 95% CI, 0.614-0.718; P < .001). For a fixed specificity of 90% (FIB4 = 1.93), the sensitivity in identifying advanced fibrosis was only 50% (95% CI, 46-55). A FIB4 > or = 2.67 had an 80% positive predictive value and a FIB4 index < or = 1.30 had a 90% negative predictive value.

CONCLUSIONS

The FIB4 index is superior to 7 other noninvasive markers of fibrosis in patients with NAFLD; however its performance characteristics highlight the need for even better noninvasive markers.

APACHE: an evaluation

A cursory evaluation of the Acute Physiology and Chronic Health Evaluation, commonly known as the APACHE scoring system, validates its relevancy as the most widely used method for assessing severity and prognosis in intensive care unit patients. The APACHE system works and the evolution from APACHE I to APACHE IV reveal that each version has its positives and negatives. It would behoove critical care nurses to know differences and how each could be best utilized.

The variation in risk adjusted mortality of intensive care units

BACKGROUND

This study aimed to estimate risk adjusted mortality rate in the ICUs (Intensive care units) by APACHE (Acute Physiology And Chronic Health Evaluation) III for revealing the performance variation in ICUs.

METHODS

This study focused on 1,090 patients in the ICUs of 18 hospitals. For establishing risk adjusted mortality predictive model, logistic regression analysis was performed. APACHE III, surgery experience, admission route, and major disease categories were used as independent variables. The performance of each model was evaluated by c-statistic and goodness-of-fit test of Hosmer-Lemeshow. Using this predictive model, the performance of each ICU was tested as ratio of predictive mortality rate and observed mortality rate.

RESULTS

The average observed mortality rate was 24.1%. The model including APACHE III score, admission route, and major disease categories was signified as the fittest one. After risk adjustment, the ratio of predictive mortality rate and observed mortality rate was distributed from 0.49 to 1.55.

CONCLUSIONS

The variation in risk adjusted mortality among ICUs was wide. The effort to reduce this quality difference is needed.

[Prognostic capacity of brain herniation signs in patients with structural neurological injury]

OBJECTIVE

To determine whether the usual mortality prediction systems (APACHE and SAPS) can be complemented by cranial computed tomography (CT) brain herniation findings in patients with structural neurological involvement.

DESIGN

Prospective cohort study.

SETTING

Trauma ICU in university hospital.

PATIENTS

One hundred and fifty five patients admitted to ICU in 2003 with cranial trauma or acute stroke.

MAIN VARIABLES OF INTEREST

Data were collected on age, diagnosis, mortality, admission cranial CT findings and on APACHE II, APACHE III and SAPS II scores.

RESULTS

Mean age was 47.8 +/- 19.4 years; APACHE II, 17.1 +/- 7.2 points; SAPS II, 43.7 +/- 17.7 points; and APACHE III, 55.8 +/- 29.7 points. Hospital mortality was 36% and mortality predicted by SAPS II was 38%, by APACHE II 30% and by APACHE III 36%. The 56 non-survivors showed greater midline shift on cranial CT scan versus survivors (4.2 +/- 5.5 vs. 1.6 +/- 3.22 mm, p = 0.002) and higher severity as assessed by SAPS II, APACHE II and APACHE III. The mortality rate was significantly higher in patients with subfalcial herniation (61% vs. 30%, p < 0.001). In the multivariate logistic regression analysis, hospital mortality was associated with the likelihood of death according to APACHE III (OR 1.07; 95% CI: 1.05-1.09) and with presence of subfalcial herniation (OR 3.15; 95% CI: 1.07-9.25).

CONCLUSIONS

In critical care patients with structural neurological involvement, cranial CT signs of subfalcial herniation complement the prognostic information given by the usual severity indexes.

[Ulysses network: an approach to integral post-ICU treatment of patients with multiple organ dysfunction syndrome]

The concept of continuity of care by intensivists as an element of quality control in the medical care of Intensive Care Unit (ICU) patients surviving multiple organ dysfunction syndrome has led to a rethinking of the ICU model in recent years. We discuss the rationale to design and implement a hospital-based, prospective, randomized, multicenter Intervention/Control study in order to estimate the impact of an interdisciplinary intervention during the post-ICU recovery phase on medium-term medical outcomes in ICU patients with multiple organ dysfunction.

Severity of Illness and Organ Failure Assessment in Adult Intensive Care Units

The critical care community has been using severity and organ failure assessment tools for over 2 decades. The major adult severity assessment models are Acute Physiology and Chronic Health Evaluation, Simplified Acute Physiology Score, and Mortality Probability Model. All three recent versions of these models perform well in predicting hospital mortality. Sequential Organ Failure Assessment score is the most used tool for assessment of multiple organ failure. These tools have been used extensively in clinical research involving critically ill patients and for benchmarking and the measurement of performance improvement. Their roles as clinical decision support tools at the bedside await future studies because of their unknown or poor performance at the individual patient level.

Analysis of physiologic alterations in intensive care unit patients and their relationship with mortality

PURPOSE

To analyze patient physiologic alterations (events) and multiple organ failure during intensive care unit (ICU) stay and examine their relationship with ICU mortality.

MATERIAL AND METHODS

A total of 17598 consecutive patients were studied for 10 months (1997-1998) in 55 European ICUs (EURICUS-II). Hourly data were collected on critical and noncritical systolic blood pressure, heart rate, oxygen saturation, and urinary events throughout ICU stay. Sepsis-related Organ Failure Assessment (SOFA) score was collected daily (6409 patients).

RESULTS

SAPS-II was 31.2 +/- 18.4 and ICU mortality 13.9%. There were 3.4 +/- 9.2 noncritical (duration, 3.9 +/- 11.4 hours) and 2 +/- 7.5 critical (3.8 +/- 13.1 hours) systolic blood pressure events per patient. Heart rate, oxygen saturation, and urinary events had similar values. Nonsurvivors had significantly more and longer physiologic alterations vs survivors. Mortality was significantly related to mean daily duration of events and mean and maximum daily SOFA. Discrimination capacity to predict ICU mortality was measured using various models: with SAPS II, area under the receiver operating characteristic curve was 0.80; with APACHE III-classified diagnosis added, 0.84; with mean duration of events/ICU day, 0.91; and with mean and maximum SOFA scores, 0.95.

CONCLUSION

Routinely gathered ICU data on physiologic variables and multiple organ failure can offer considerable complementary information not provided by usual mortality prediction systems; and their weight in daily care policy decisions may need to be revisited.

Prognosis in critical care

Prognostic risk prediction models have been employed in the intensive care unit (ICU) setting since the 1980s and provide health care providers with important information to help inform decisions related to treatment and prognosis, as well as to compare outcomes across institutions. Prognostic models for critical care are among the most widely utilized and tested predictive models in healthcare. In this article, we review and compare mortality prediction models, including the APACHE (1981), SAPS (1984), APACHE-II (1985), MPM (1987), APACHE-III (1991), SAPS-II (1993), and MPM-II (1993). We emphasize the importance of model calibration in this domain. In addition, we present a brief review of the statistical methodology, multiple logistic regression, which underlies most of the models currently used in critical care.

Six-year mortality and quality of life in critically ill patients with chronic obstructive pulmonary disease

OBJECTIVE

To study the mortality and quality of life (QOL) of survivors at 6 yrs after intensive care unit (ICU) admission for chronic obstructive pulmonary disease.

DESIGN

Prospective, multiple-center cohort study.

SETTING

A total of 86 ICUs throughout Spain.

PATIENTS

Patients in the Project for the Epidemiological Analysis of Critical Care Patients (PAEEC) project with chronic obstructive pulmonary disease were included.

MEASUREMENTS AND MAIN RESULTS

The sample comprised 742 patients; 508 of them were admitted for acute exacerbation of chronic obstructive pulmonary disease, and 379 of these required intermittent positive-pressure ventilation. The mean age of the patients was 65.2 +/- 9.89 yrs, Acute Physiology and Chronic Health Evaluation (APACHE) III score was 66.6 +/- 21.04; preadmission QOL questionnaire score was 7 +/- 4.82 points, and hospital mortality was 31.8%. At 6 yrs, 32.2% had died after hospital discharge, 21.6% could not be traced, and 107 patients were alive (18.3% of the 582 followed-up patients). QOL of survivors was worse than preadmission (6.55 +/- 5.6 vs. 4.92 +/- 4.5 points, p < .05), but 72% of patients were self-sufficient. Among the 379 patients admitted to the ICU for acute chronic obstructive pulmonary disease exacerbation and requiring intermittent positive-pressure ventilation, 36.7% died in the hospital; at 6 yrs after hospital discharge, 31.4% had died, 18.7% could not be traced, and 50 patients (16.2% of followed-up patients) were alive. Multivariate analysis with logistic regression showed that the mortality at 6 yrs was related to age (odds ratio, 1.046; 95% confidence interval, 1.023-1.071), APACHE III score (odds ratio, 1.013; 95% confidence interval, 1.001-1.024), and preadmission QOL score (odds ratio, 1.139; 95% confidence interval, 1.078-1.204).

CONCLUSION

The 6-yr mortality of patients with chronic obstructive pulmonary disease requiring ICU admission is high. Mortality is mainly influenced by pre-ICU admission QOL. At 6 yrs, at least 15% are alive; survivors have a worse QOL compared with pre-ICU admission, although three quarters of them are self-sufficient.

Costes de la calidad en Medicina Intensiva. Guía para gestores clínicos

OBJECTIVE

This article reviews the utility and applicability of available systems in order to calculate general and quality costs in clinical services settings.

METHODS

Review of techniques to calculate costs in Intensive Care Units (ICUs) according to analytical accounting approaches.

RESULTS

The methodological development is complemented with the results of its application in the ICU of the Miracle's Hospital showing the structure of costs and the results obtained with this methodology when analyzing the costs of activities related to quality improvement.

CONCLUSIONS

The effort to implement systems focused to analyze general and quality costs will result in a benefit of those participating in the healthcare system: citizens, professionals, managers, and "financials" since that which is only a legitimate demand today will be a inexcuseable commitment of the healthcare professionals from the society tomorrow.

Recalibration of risk prediction models in a large multicenter cohort of admissions to adult, general critical care units in the United Kingdom*

OBJECTIVE

To assess the performance of published risk prediction models in common use in adult critical care in the United Kingdom and to recalibrate these models in a large representative database of critical care admissions.

DESIGN

Prospective cohort study.

SETTING

A total of 163 adult general critical care units in England, Wales, and Northern Ireland, during the period of December 1995 to August 2003.

PATIENTS

A total of 231,930 admissions, of which 141,106 met inclusion criteria and had sufficient data recorded for all risk prediction models.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The published versions of the Acute Physiology and Chronic Health Evaluation (APACHE) II, APACHE II UK, APACHE III, Simplified Acute Physiology Score (SAPS) II, and Mortality Probability Models (MPM) II were evaluated for discrimination and calibration by means of a combination of appropriate statistical measures recommended by an expert steering committee. All models showed good discrimination (the c index varied from 0.803 to 0.832) but imperfect calibration. Recalibration of the models, which was performed by both the Cox method and re-estimating coefficients, led to improved discrimination and calibration, although all models still showed significant departures from perfect calibration.

CONCLUSIONS

Risk prediction models developed in another country require validation and recalibration before being used to provide risk-adjusted outcomes within a new country setting. Periodic reassessment is beneficial to ensure calibration is maintained.

Do we have reliable biochemical markers to predict the outcome of critical illness?

Current outcome prediction in critically ill patients relies on the art of clinical judgement and/or the science of prognostication using illness severity scores. The biochemical processes underlying critical illness have increasingly been unravelled. Several biochemical markers reflecting the process of inflammation, immune dysfunction, impaired tissue oxygenation and endocrine alterations have been evaluated for their predictive power in small subpopulations of critically ill patients. However, none of these parameters has been validated in large populations of unselected ICU patients as has been done for the illness severity and organ failure scores. A simple biochemical predictor of ICU mortality will probably remain elusive because the processes underlying critical illness are very complex and heterogeneous. Future prognostic models will need to be far more sophisticated.

Predicting the clinical outcome of tetanus: the tetanus severity score

OBJECTIVES

To create a new tetanus score and compare it with the Phillips and Dakar scores.

METHODS

We used prospectively acquired data from consecutive patients admitted to the Hospital for Tropical Diseases, Ho Chi Minh City, to create the Tetanus Severity Score (TSS) with multivariate logistic regression. We compared the new score with Phillips and Dakar scores by means of resubstituted and prospective data, assessing performance in terms of sensitivity, specificity and area under receiver operator characteristic curves.

RESULTS

Resubstitution testing yielded a sensitivity of 77% (298/385) and a specificity of 82% (1,183/1,437) for the TSS; 89% (342/385) and 20% (281/1,437) for the Phillips score; and 13% (49/385) and 98% (1,415/1,437) for the Dakar score. The TSS showed greatest discrimination with 0.89 area under the receiver operator characteristic curve (95% CI 0.88-0.90); this was 0.74 for the Dakar score and (95% CI 0.71-0.77) and 0.66 for the Phillips score (95% CI 0.63-0.70; P values <0.001). Prospective testing showed 65% (13/20) sensitivity and 91% (210/230) specificity for the TSS; 80% (16/20) and 51% (118/230) for the Phillips score; and 25% (5/20) and 96% (221/230) for the Dakar score. The TSS achieved the greatest area under TSS of 0.89 (95% CI 0.82-0.96), significantly greater than the Phillips score [0.74 (0.6-0.88), P = 0.049] but not the Dakar score [0.80, (0.71-0.90), P = 0.090].

CONCLUSIONS

The TSS is the first prospectively developed classification scheme for tetanus and should be adopted to aid clinical triage and management and as a basis for clinical research.

Incidence of acute respiratory distress syndrome and its relation to age

PURPOSE

The incidence of acute respiratory distress syndrome (ARDS) was previously considered to be relatively low, at less than 10 cases per 100,000 inhabitants per year, but recent reports suggest a higher incidence, especially in elderly patients. The objective was to determine the incidence and mortality of ARDS in our setting, both overall and by age group.

MATERIALS AND METHODS

We conducted a prospective, observational study of patients older than 14 years, admitted to the intensive care units of all hospitals in a province of southern Spain (Granada) during a 5-month period in 2001. American-European Consensus Conference criteria for ARDS were used. Patients were divided into 5 age groups, and the hospital mortality was recorded.

RESULTS

During the study period, 61 Granada-residing patients developed ARDS criteria. This represents an overall incidence of 23 cases per 100,000 inhabitants per year in the province. The incidence of ARDS in the age groups of 15 to 29, 30 to 44, 45 to 59, 60 to 74, and older than 74 years was 4.6, 13.6, 21.6, 51, and 73.9 cases per 100,000 inhabitants per year, respectively. The overall hospital mortality rate was 66%.

CONCLUSIONS

The incidence of ARDS is higher than reported a decade ago and is especially elevated in the elderly. The mortality remains high.

SAPS 3--From evaluation of the patient to evaluation of the intensive care unit. Part 1: Objectives, methods and cohort description

OBJECTIVE

Risk adjustment systems now in use were developed more than a decade ago and lack prognostic performance. Objective of the SAPS 3 study was to collect data about risk factors and outcomes in a heterogeneous cohort of intensive care unit (ICU) patients, in order to develop a new, improved model for risk adjustment.

DESIGN

Prospective multicentre, multinational cohort study.

PATIENTS AND SETTING

A total of 19,577 patients consecutively admitted to 307 ICUs from 14 October to 15 December 2002.

MEASUREMENTS AND RESULTS

Data were collected at ICU admission, on days 1, 2 and 3, and the last day of the ICU stay. Data included sociodemographics, chronic conditions, diagnostic information, physiological derangement at ICU admission, number and severity of organ dysfunctions, length of ICU and hospital stay, and vital status at ICU and hospital discharge. Data reliability was tested with use of kappa statistics and intraclass-correlation coefficients, which were >0.85 for the majority of variables. Completeness of the data was also satisfactory, with 1 [0-3] SAPS II parameter missing per patient. Prognostic performance of the SAPS II was poor, with significant differences between observed and expected mortality rates for the overall cohort and four (of seven) defined regions, and poor calibration for most tested subgroups.

CONCLUSIONS

The SAPS 3 study was able to provide a high-quality multinational database, reflecting heterogeneity of current ICU case-mix and typology. The poor performance of SAPS II in this cohort underscores the need for development of a new risk adjustment system for critically ill patients.

Epidemiology and natural history of acute renal failure in the ICU

This article focuses on the incidence, risk factors, and mortality of acute renal failure (ARF) in critically ill patients. Accurate epidemiologic assessment of ARF is still a problem; as long as there is neither a uniquely accepted definition of ARF nor definitions for end points to measure, results will remain heterogeneous and hard to compare. Mortality of ARF has remained high throughout the last decades, despite further development of modern treatment modalities. This indicates that ARF is not just a matter of loss of organ function that can easily be replaced easily by extracorporeal therapies, but is a condition additionally accompanied by systemic consequences which significantly impact on prognosis.

Use of the Sequential Organ Failure Assessment score as a severity score

OBJECTIVE

To evaluate whether the SOFA score can be used to develop a model to predict intensive care unit (ICU) mortality in different countries.

DESIGN AND SETTING

Analysis of a prospectively collected database. Patients with ICU stay longer than 2 days were studied to develop a mortality prediction model based on measurements of organ dysfunction.

PATIENTS

748 patients from six countries.

MEASUREMENTS AND RESULTS

Two logistic regression models were constructed, one based on the SOFA maximum (SOFA Max model) and the other on variables identified by multivariate regression (SOFA Max-infection model). The H and C statistics had a p value above 0.05 for both models, but the D statistics showed a poor performance on the SOFA Max model when stratified for the presence of infection. Subsequent analysis was performed with SOFA Max-infection model. The area under the curve was 0.853. There were no statistically significant differences in observed and predicted mortalities except for one country which had a higher than predicted ICU mortality both in the overall population (28.3 vs. 19.1%) and in the noninfected patients (21.4 vs. 12.6%).

CONCLUSIONS

The SOFA Max adjusted for age and the presence of infection can predict mortality in this population, but in one country the ICU mortality was higher than expected. Our data do not allow us to determine the reasons behind these differences, and further studies to detect differences in mortality between countries and to elucidate the basis for these differences should be encouraged.

Using hierarchical modeling to measure ICU quality

OBJECTIVE

To determine whether hierarchical modeling agrees with conventional logistic regression modeling on the identity of ICU quality outliers within a large multi-institutional database.

DESIGN

Retrospective database analysis.

SETTING AND PATIENTS

Subset of the Project IMPACT database consisting of 40435 adult patients admitted to surgical, medical, and mixed surgical-medical ICUs ( n=55) between 1997 and 1999 who met inclusion criteria for SAPS II.

MEASUREMENTS AND RESULTS

The SAPS II score was customized to this database using conventional logistic regression and using a hierarchical (random coefficients) model. Both models exhibited excellent discrimination ( Cstatistic) and calibration (Hosmer-Lemeshow statistic). The hierarchical and nonhierarchical models had C statistics of.870 and.865, and HL statistics of 3.71 ( p>.88, df=8) and 8.94 ( p>.35, df=8), respectively. Since the random effects component of the hierarchical model accounts for between-hospital variability, only the fixed-effects coefficients were used to calculate the expected mortality rate based on the hierarchical model. The ratio and 95% confidence intervals of the observed to expected mortality rate were calculated using both models for each ICU. ICUs whose observed/expected ratio was either less than 1 or greater than 1, and whose 95% confidence interval did not include 1 were labeled as either high-performance or low-performance outliers, respectively. Analysis using kappa statistic revealed almost perfect agreement between the two models (nonhierarchical vs. hierarchical) on the identity of ICU quality outliers.

CONCLUSIONS

Models obtained by customizing SAPS II using a nonhierarchical and a hierarchical approach exhibit excellent agreement on the identity of ICU quality outliers.

Long-term outcome in ICU patients: what about quality of life?

OBJECTIVE

Analysis of mortality and quality of life (QOL) after intensive care unit (ICU) discharge.

DESIGN

Prospective, observational study.

SETTING

Mixed, 31-bed, medico-surgical ICU.

PATIENTS

Consecutive adult ICU admissions between June 25 and September 10, 2000, except admissions for uncomplicated elective postoperative surveillance. INTERVENTIONS. None.

MEASUREMENTS AND RESULTS

Age, past history, admission APACHE II, SOFA score (admission, maximum, discharge), ICU and hospital mortality were recorded. A telephone interview employing the EuroQol 5D system was conducted 18 months after discharge. Of 202 patients, 34 (16.8%) died in the ICU and 23 (11.4%) died in the hospital after ICU discharge. Of the 145 patients discharged alive from hospital, 22 could not be contacted and 27 (13.4%) had died after hospital discharge. Of the 96 patients (47.5%) who completed the questionnaire, 38% had a worse QOL than prior to ICU admission, but only 8.3% were severely incapacitated. Twenty-three patients (24%) had reduced mobility, 15 (15.6%) had limited autonomy, 24 (25%) had alteration in usual daily activities, 29 (30.2%) expressed more anxiety/depression, and 42 (44%) had more discomfort or pain. Twenty-eight (62.2% of those who worked previously) patients had returned to work 18 months after ICU discharge.

CONCLUSIONS

Comparing QOL after discharge with that before admission, patients more frequently report worse QOL for the domains of pain/discomfort and anxiety/depression than for physical domains. Factors commonly associated with a change in QOL were previous problems in the affected domains, prolonged hospital length of stay (LOS), greater disease severity at admission and degree of organ dysfunction during ICU stay.

Automated intensive care unit risk adjustment: results from a National Veterans Affairs study

CONTEXT

Comparison of outcome among intensive care units (ICUs) requires risk adjustment for differences in severity of illness and risk of death at admission to the ICU, historically obtained by costly chart review and manual data entry.

OBJECTIVE

To accurately estimate patient risk of death in the ICU using data easily available in hospital electronic databases to permit automation.

DESIGN AND SETTING

Cohort study to develop and validate a model to predict mortality at hospital discharge using multivariate logistic regression with a split derivation (17,731) and validation (11,646) sample formed from 29,377 consecutive first ICU admissions to medical, cardiac, and surgical ICUs in 17 Veterans' Health Administration hospitals between February 1996 and July 1997.

MAIN OUTCOME MEASURES

Mortality at hospital discharge adjusted for age, laboratory data, diagnosis, source of ICU admission, and comorbid illness.

RESULTS

The overall hospital death rate was 11.3%. In the validation sample, the model separated well between survivors and nonsurvivors (area under the receiver operating characteristic curve = 0.885). Examination of the observed vs. the predicted mortality across the range of mortality showed the model was well calibrated.

CONCLUSIONS

Automation could broaden access to risk adjustment of ICU outcomes with only a small trade-off in discrimination. Broader use might promote valid evaluation of ICU outcomes, encouraging effective practices and improving ICU quality.

External validation of the SAPS II, APACHE II and APACHE III prognostic models in South England: a multicentre study

OBJECTIVE

External validation of three prognostic models in adult intensive care patients in South England. DESIGN. Prospective cohort study.

SETTING

Seventeen intensive care units (ICU) in the South West Thames Region in South England.

PATIENTS AND PARTICIPANTS

Data of 16646 patients were analysed.

INTERVENTIONS

None.

MEASUREMENTS AND RESULTS

We compared directly the predictive accuracy of three prognostic models (SAPS II, APACHE II and III), using formal tests of calibration and discrimination. The external validation showed a similar pattern for all three models tested: good discrimination, but imperfect calibration. The areas under the receiver operating characteristics (ROC) curves, used to test discrimination, were 0.835 and 0.867 for APACHE II and III, and 0.852 for the SAPS II model. Model calibration was assessed by Lemeshow-Hosmer C-statistics and was Chi(2 )=232.1 for APACHE II, Chi(2 )=443.3 for APACHE III and Chi(2 )=287.5 for SAPS II.

CONCLUSIONS

Disparity in case mix, a higher prevalence of outcome events and important unmeasured patient mix factors are possible sources for the decay of the models' predictive accuracy in our population. The lack of generalisability of standard prognostic models requires their validation and re-calibration before they can be applied with confidence to new populations. Customisation of existing models may become an important strategy to obtain authentic information on disease severity, which is a prerequisite for reliably measuring and comparing the quality and cost of intensive care.

Simplified Acute Physiology Score III: a project for a new multidimensional tool for evaluating intensive care unit performance

The Simplified Acute Physiology Score III Outcomes Research Group is developing an international multidimensional instrument for the global evaluation of intensive care unit performance. Among its specific objectives are the update of a severity of illness index (Simplified Acute Physiology Score) with a mortality prediction equation, with the hindsight of recent years, and the creation or application of novel instruments in the areas of infections and cost-effectiveness. Some important measurements such as the quality of life and the satisfaction of patients and professionals are not included. A further aim is the achievement of international validation.

The effects of two methods for customising the original SAPS II model for intensive care patients from South England

Model customisation is used to adjust prognostic models by re-calibrating them to obtain more reliable mortality estimates. We used two methods for customising the Simplified Acute Physiology Score II model for 15,511 intensive care patients by altering the logit and the coefficients of the original equation. Both methods significantly improved model calibration, but customising the coefficients was slightly more effective. The Hosmer-Lemeshow chi(2)-value improved from 306.0 (p< 0.001) before, to 14.5 (p < 0.07) and 23.3 (p < 0.06) after customisation of the coefficients and the logit, respectively. Discrimination was not affected. The standardised mortality ratio for the entire population declined from 1.16 (95% confidence interval: 1.13-1.20, p < 0.001) to 0.99 (95% confidence interval: 0.96-1.02, p < 0.22) after customisation of the coefficients. The uniformity-of-fit for patients grouped by operative status and comorbidities also improved, but remained imperfect for patients stratified by location before intensive care unit admission. Amalgamation of large, regional databases could provide the basis for the re-calibration of standard prognostic models, which could then be used as a national reference system to allow more reliable comparisons of the efficacy and quality of care based on severity adjusted outcome measures.

A cross-cultural comparison of critical care delivery: Japan and the United States

OBJECTIVE

To compare the utilization and outcomes of critical care services in a cohort of hospitals in the United States and Japan.

DESIGN

Prospective data collection on 5,107 patients and detailed organizational characteristics from each of the participating Japanese study hospitals between 1993 and 1995, with comparisons made to prospectively collected data on the 17,440 patients included in the US APACHE (acute physiology and chronic health evaluation) III database.

SETTING

Twenty-two Japanese and 40 US hospitals.

PATIENTS

Consecutive, unselected patients from medical, surgical, and mixed medical/surgical ICUs.

MEASUREMENTS

Severity of illness, predicted risk of in-hospital death, and ICU and hospital length of stay (LOS) were assessed using APACHE III. Japanese ICU directors completed a detailed survey describing their units.

MAIN RESULTS

US and Japanese ICUs have a similar array of modalities available for care. Only 1.0% (range, 0.56 to 2.7%) of beds in Japanese hospitals were designated as ICUs. The organization of the Japanese and US ICUs varied by hospital, but Japanese ICUs were more likely to be organized to care for heterogeneous diagnostic populations. Sample case-mix differences reflect different disease prevalence. ICU utilization for women is significantly lower (35.5% vs 44.8% of patients) and there were relatively fewer patients > or = 85 years old in the Japanese ICU cohort (1.2% vs 4.6%), despite a higher per capita rate of individuals > or = 85 years old in Japan. The utilization of ICUs for patients at low risk of death significantly less in Japan (10.2%) than in the United States (12.9%). The APACHE III score stratified patient risk. Overall mortality was similar in both national samples after accounting for differences in hospital LOS, utilizing a model that was highly discriminating (receiver operating characteristic, 0.87) when applied to the Japanese sample. The application of a US-based mortality model to a Japanese sample overestimated mortality across all but the highest (> 90%) deciles of risk. Significant variation in expected performance was noted between hospitals. Risk-adjusted ICU LOS was not significantly longer in Japan; however, total hospital stay was nearly twice that found in the US hospitals, reflecting differences in hospital utilization philosophies.

CONCLUSIONS

Similar high-technology critical care is available in both countries. Variations in ICU utilization reflect differences in case-mix and bed availability. Japanese ICU utilization by gender reflects differences in disease prevalence, whereas differences in utilization by age may reflect differences in cultural norms regarding the limits of care. Such differences provide context from which to assess the delivery of care across international borders. Miscalibration of predictive models applied to international data samples highlight the impact that differences in resource use and local practice cultures have on outcomes. Models may require modification in order to account for these differences. Nevertheless, with large databases, it is possible to assess critical care delivery systems between countries accounting for differences in case-mix, severity of illness, and cultural normative standards facilitating the design and management such systems.