Prediction Models and Their External Validation Studies for Mortality of Patients with Acute Kidney Injury: A Systematic Review

External validation of a predictive model for post-treatment persistent disease by 131I whole-body scintigraphy in patients with differentiated thyroid cancer

PURPOSE

We performed an external validation of a predictive model for persistent/metastatic disease in patients with differentiated thyroid cancer (DTC) at post-treatment 131I whole-body scintigraphy (WBS).

METHODS

Our study population included 836 patients (median age 44 years, 78% women) with DTC referred from 1994 to 2021 at our center. Age, sex, histology, T stage, N stage, American Thyroid Association risk classes, thyroid-stimulating hormone, radioactive iodine (RAI) activity, and thyroglobulin (Tg) levels were considered potential predictors of post-treatment WBS results. For the external validation, N stage and Tg levels were put into the decision tree (DT) model using its same Tg cut-off values.

RESULTS

Ninety-nine patients (12%) had positive post-treatment WBS. The area under receiver operating characteristic (ROC) curve for predicting WBS findings through the external validation was 0.60 (95% confidence interval, CI, 0.56-0.64), and positive and negative predictive values were 58% (95% CI, 41-74%) and 90% (95% CI, 88-92%). We also developed an internal model including the independent predictors of WBS findings (i.e., Tg levels, T stage, N stage, and RAI activity). For this model the area under ROC curve was 0.75 (95% CI, 0.69-0.81), and positive and negative predictive values were 90% (95% CI, 68-99% and 88-92%).

CONCLUSIONS

The external validation of the proposed DT model has a limited value for predicting post-treatment 131I-WBS findings in our patients. The internal model including also T stage and RAI activity demonstrates higher predictive value.

Development and validation of a machine-learning model for predicting the risk of death in sepsis patients with acute kidney injury

The mortality rate of patients with sepsis-induced acute kidney injury (S-AKI) is notably elevated. The initial categorization of prognostic indicators has a beneficial impact on elucidating and enhancing disease outcomes. This study aimed to predict the mortality risk of S-AKI patients by employing machine learning techniques. The sample size determined by a four-step procedure yielded 1508 samples. The research design necessitated the inclusion of individuals with S-AKI from the Medical Information Mart for Intensive Care (MIMIC)-IV database. The patients were initially admitted to the Intensive Care Unit (ICU) for their hospital stay. Additionally, these patients (aged from 18 to 89 years old) had encountered S-AKI on the day of their admittance. Forty-two predictive factors were analyzed, with hospitalization death as the outcome variable. The training set (4001 cases) consisted of 70 % of the participants, and the remaining (1714 cases) participants were allocated to the validation set. Furthermore, an additional validation set (MIMIC-III) consisted of 1757 patients from the MIMIC-III database. Moreover, an external validation set from the Intensive Care Department of Beijing Friendship Hospital (BFH) comprised 72 patients. Six machine learning models were employed in the prediction, namely the logistic, lasso, rpart, random forest, xgboost, and artificial neural network models. The comparative efficacy of the newly developed model in relation to the APACHE II model for predicting mortality risk was also assessed. The XGBoost model exhibited a superior performance with the training set. With the internal validation set and the two external validation sets (MIMIC-III and BFH), the xgboost algorithm demonstrated the highest performance. Meanwhile, APACHE II performed poorly at predicting the mortality risk with the BFH validation set. The mortality risk was influenced by three primary clinical parameters: urine volume, lactate, and Glasgow Coma Scale (GCS) score. Thus, we developed a prediction model for the risk of death among S-AKI patients that has an improved performance compared to previous models and is a potentially valuable tool for S-AKI prediction and treatment in the clinic.

Association between total bilirubin/Albumin ratio and all-cause mortality in acute kidney injury patients: A retrospective cohort study

BACKGROUND

The association between the total bilirubin/albumin (B/A) and the all-cause mortality of critically ill patients with acute kidney injury (AKI) remains unclear. This retrospective study aimed to investigate the relationship between B/A ratio and mortality in patients with AKI.

METHODS

The clinical data of AKI patients in the Medical Information Mart for Intensive Care III (MIMIC-III) database were retrospectively analyzed. Patients were divided into the low and high B/A groups (B/A ≤ 0.25 and B/A > 0.25, respectively). The primary outcome was 28-day all-cause mortality, and the secondary outcomes were 60-day, 1-year and 4-year all-cause mortality. Kaplan-Meier survival curves and Cox proportional risk models were constructed to evaluate the effect of B/A on survival outcomes.

RESULTS

The 28-day mortality rates were 18.00% and 25.10% in the low and high B/A groups, respectively (P < 0.001). The Kaplan-Meier analysis showed that patients with higher B/A values had higher all-cause mortality risk (log-rank P < 0.0001). The multivariate Cox proportional risk analysis showed that B/A was an independent risk predictor for death at 28 days, 60 days, 1 year, and 4 years.

CONCLUSION

B/A is an independent risk factor for increased mortality in patients with AKI and may be used as a predictor of clinical outcomes in AKI.

Development and validation of clinical prediction models for acute kidney injury recovery at hospital discharge in critically ill adults

PURPOSE

Acute kidney injury (AKI) recovery prediction remains challenging. The purpose of the present study is to develop and validate prediction models for AKI recovery at hospital discharge in critically ill patients with ICU-acquired AKI stage 3 (AKI-3).

METHODS

Models were developed and validated in a development cohort (n = 229) and a matched validation cohort (n = 244) from the multicenter EPaNIC database to create prediction models with the least absolute shrinkage and selection operator (Lasso) machine-learning algorithm. We evaluated the discrimination and calibration of the models and compared their performance with plasma neutrophil gelatinase-associated lipocalin (NGAL) measured on first AKI-3 day (NGAL_AKI3) and reference model that only based on age.

RESULTS

Complete recovery and complete or partial recovery occurred in 33.20% and 51.23% of the validation cohort patients respectively. The prediction model for complete recovery based on age, need for renal replacement therapy (RRT), diagnostic group (cardiac/surgical/trauma/others), and sepsis on admission had an area under the receiver operating characteristics curve (AUROC) of 0.53. The prediction model for complete or partial recovery based on age, need for RRT, platelet count, urea, and white blood cell count had an AUROC of 0.61. NGAL_AKI3 showed AUROCs of 0.55 and 0.53 respectively. In cardiac patients, the models had higher AUROCs of 0.60 and 0.71 than NGAL_AKI3's AUROCs of 0.52 and 0.54. The developed models demonstrated a better performance over the reference models (only based on age) for cardiac surgery patients, but not for patients with sepsis and for a general ICU population.

CONCLUSION

Models to predict AKI recovery upon hospital discharge in critically ill patients with AKI-3 showed poor performance in the general ICU population, similar to the biomarker NGAL. In cardiac surgery patients, discrimination was acceptable, and better than NGAL. These findings demonstrate the difficulty of predicting non-reversible AKI early.

Platelet Function in Acute Kidney Injury: A Systematic Review and a Cohort Study

Acute kidney injury (AKI) patients have increased bleeding risk, which could be partially due to acquired platelet dysfunction. We conducted a systematic review and a cohort study to investigate platelet function and count in AKI and their association with AKI-related bleeding and mortality. Through a systematic literature search in PubMed and Embase, we identified 9 studies reporting platelet function and 56 studies reporting platelet count or platelet indices in AKI patients. Overall, platelet aggregation was reduced in AKI patients in nonintensive care unit (ICU) settings but not in ICU settings, except that reduced aggregation was associated with renal replacement therapy. Thrombocytopenia in AKI was frequent and often predictive of mortality. In our cohort study, we prospectively included 54 adult ICU patients who developed AKI within 24 hours of ICU admission and 33 non-AKI ICU controls. Platelet function was measured with light transmission aggregometry and flow cytometry. AKI patients bled more frequently than non-AKI patients (p = 0.04), and bleeding was associated with increased 30-day mortality in AKI (p = 0.02). However, platelet function was not different between AKI and non-AKI patients (aggregation: all p > 0.52; flow cytometry: all p > 0.07) and platelet function was not associated with bleeding in AKI. In conclusion, a reduced platelet count is frequent in AKI, but the literature on platelet function in AKI is sparse. In a cohort study, we demonstrated that patients with AKI within 24 hours of ICU admission exhibited increased bleeding tendency but this was not associated with reduced platelet function.

Prediction Models for One-Year Survival of Adult Patients with Acute Kidney Injury: A Longitudinal Study Based on the Data from the Medical Information Mart for Intensive Care III Database

Acute kidney injury (AKI) is a common complication of acute illnesses with unfavorable outcomes. This cohort study aimed at constructing prediction models for one-year survival in adult AKI patients based on prognostic nutritional index (PNI), platelet-to-lymphocyte ratio (PLR), neutrophil percentage-to-albumin ratio (NPAR), or neutrophil-to-lymphocyte ratio (NLR), respectively. In total, 6050 patients from Medical Information Mart for Intensive Care III (MIMIC-III) were involved. The least absolute shrinkage and selection operator (LASSO) regression was utilized to screen possible covariates. The samples were randomly divided into the training set and the testing set at a ratio of 7.5 : 2.5, and the prediction models were constructed in the training set by random forest. The prediction values of the models were measured via sensitivity, specificity, negative prediction value (NPV), positive prediction value (PPV), area under the curve (AUC), and accuracy. We found that NLR (OR = 1.261, 95% CI: 1.145–1.388), PLR (OR = 1.295, 95% CI: 1.152–1.445), and NPAR (OR = 1.476, 95% CI: 1.261–1.726) were associated with an increased risk, while PNI (OR = 0.035, 95% CI: 0.020–0.059) was associated with a decreased risk of one-year mortality in AKI patients. The AUC was 0.964 (95% CI: 0.959–0.969) in the training set based on PNI, age, gender, length of stay (LOS) in hospital, platelets (PLT), ethnicity, LOS in ICU, systolic blood pressure (SBP), diastolic blood pressure (DBP), heart rate, glucose, AKI stage, atrial fibrillation (AF), vasopressor, renal replacement therapy (RRT), and mechanical ventilation. The testing set was applied as the internal validation of the model with an AUC of 0.778 (95% CI: 0.754–0.801). In conclusion, PNI accompanied by age, gender, ethnicity, SBP, DBP, heart rate, PLT, glucose, AF, RRT, mechanical ventilation, vasopressor, AKI stage, LOS in ICU, and LOS in hospital exhibited a good predictive value for one-year mortality of AKI patients.

Application of interpretable machine learning for early prediction of prognosis in acute kidney injury

Background

This study aimed to develop an algorithm using the explainable artificial intelligence (XAI) approaches for the early prediction of mortality in intensive care unit (ICU) patients with acute kidney injury (AKI).

Methods

This study gathered clinical data with AKI patients from the Medical Information Mart for Intensive Care IV (MIMIC-IV) in the US between 2008 and 2019. All the data were further randomly divided into a training cohort and a validation cohort. Seven machine learning methods were used to develop the models for assessing in-hospital mortality. The optimal model was selected based on its accuracy and area under the curve (AUC). The SHapley Additive exPlanation (SHAP) values and Local Interpretable Model-Agnostic Explanations (LIME) algorithm were utilized to interpret the optimal model.

Results

A total of 22,360 patients with AKI were finally enrolled in this study (median age, 69.5 years; female, 42.8%). They were randomly split into a training cohort (16770, 75%) and a validation cohort (5590, 25%). The eXtreme Gradient Boosting (XGBoost) model achieved the best performance with an AUC of 0.890. The SHAP values showed that Glasgow Coma Scale (GCS), blood urea nitrogen, cumulative urine output on Day 1 and age were the top 4 most important variables contributing to the XGBoost model. The LIME algorithm was used to explain the individualized predictions.

Conclusions

Machine-learning models based on clinical features were developed and validated with great performance for the early prediction of a high risk of death in patients with AKI.

A Novel Predictive Model for Hospital Survival in Patients who are Critically Ill with Dialysis-Dependent AKI: A Retrospective Single-Center Exploratory Study

BACKGROUND

Mortality of patients who are critically ill with AKI initiated on RRT is very high. Identifying modifiable and unmodifiable clinical variables at dialysis start that are associated with hospital survival can help, not only in prognostication, but also in clinical triaging.

METHODS

A retrospective observational study was conducted on patients with AKI-D who were initiated on RRT in the medical and surgical intensive care units (ICUs) of a high-acuity academic medical center from January 2010 through December 2015. We excluded patients with suspected poisoning, ESKD, stage 5 CKD not on dialysis, or patients with AKI-D initiated on RRT outside of the ICU setting. The primary outcome was in-hospital mortality.

RESULTS

Of the 416 patients who were critically ill with AKI-D admitted to the medical (38%), surgical (41%), and cardiac (21%) ICUs, with nearly 75% on artificial organ support, the mean age 62.1±14.8 years, mean SOFA score was 11.8±4.3, dialysis was initiated using continuous RRT in 261 (63%) and intermittent hemodialysis in 155 (37%) patients. Incidence of survival to hospital discharge was 48%. Using multivariable logistic regression with stepwise backward elimination, a prognostic model was created that included the variables age, CKD, COPD, admission, and within 24 hours of the start SOFA score, refractory hyperkalemia and uremic encephalopathy as dialysis indications, BUN >100 mg/dl, serum creatinine, serum lactate, serum albumin, CRRT as initial modality, severe volume overload, and abdominal surgery. The model exhibited good calibration (goodness of fit test, P=0.83) and excellent discrimination (optimism-corrected C statistic 0.93).

CONCLUSIONS

In this single-center, diverse, critically ill AKI-D population, a novel prognostic model that combined widely used ICU scores, clinical and biochemical data at dialysis start, and dialysis indication and modality, robustly predicted short-term survival. External validation is needed to prove the generalizability of the study findings.

Predictors for mortality and recovery in patients with acute renal injury receiving continuous renal replacement therapy

OBJECTIVE

Despite continuous renal replacement therapy (CRRT) has been widely used in critically ill patients with acute kidney injury (AKI), the prognosis and recovery of renal function in these patients are still poor. Therefore, we aimed to identify the prognostic factors for the mortality and recovery of renal function in patients with AKI receiving CRRT.

METHODS

A total of 125 patients with AKI, treated with CRRT in the emergency intensive care unit (EICU) in an academic teaching hospital from January 2014 to December 2018 were enrolled in this retrospective study. The clinical data of these patients were collected. Univariate regression analysis and multivariate regression analysis were conducted to identify the predictors for the mortality and recovery of renal function.

RESULTS

The median age was 68.0 (56.5-79.0) years old, and from which 69.6% were males. Sixty-four patients (51.2%) survived and 50 patients (40%) recovered their renal function. Multivariate regression analysis showed that the independent risk factors for mortality were male (odds ratio [OR]:3.771, 95% confidence interval [CI]:1.063-13.372, p = 0.04), Acute Physiology and Chronic Health Evaluation (APACHE) II score (OR: 1.187, 95% CI: 1.050-1.341, p = 0.006), mechanical ventilation (OR: 6.266, 95% CI: 1.771-22.167, p = 0.004) and vasopressor use (OR: 5.224, 95% CI: 1.546-17.657, p = 0.008). Moreover, the independent predictors for not recovering of renal function were male (OR: 3.440, 95% CI: 1.271-9.311, p = 0.015), pre-existing comorbidity of hypertension (OR: 4.207, 95% CI: 1.609-11.000, p = 0.003) and vasopressor use (OR: 5.280, 95% CI: 2.018-13.811, p = 0.001).

CONCLUSIONS

Male, high APACHE II score, mechanical ventilation and vasopressor use were closely associated with the increased mortality, while male, pre-existing history of hypertension, and vasopressor use were the independent predictors for non-recovery of renal function.

Integrating electronic health data records to develop and validate a predictive model of hospital-acquired acute kidney injury in non-critically ill patients

Background

Models developed to predict hospital-acquired acute kidney injury (HA-AKI) in non-critically ill patients have a low sensitivity, do not include dynamic changes of risk factors and do not allow the establishment of a time relationship between exposure to risk factors and AKI. We developed and externally validated a predictive model of HA-AKI integrating electronic health databases and recording the exposure to risk factors prior to the detection of AKI.

Methods

The study set was 36 852 non-critically ill hospitalized patients admitted from January to December 2017. Using stepwise logistic analyses, including demography, chronic comorbidities and exposure to risk factors prior to AKI detection, we developed a multivariate model to predict HA-AKI. This model was then externally validated in 21 545 non-critical patients admitted to the validation centre in the period from June 2017 to December 2018.

Results

The incidence of AKI in the study set was 3.9%. Among chronic comorbidities, the highest odds ratios (ORs) were conferred by chronic kidney disease, urologic disease and liver disease. Among acute complications, the highest ORs were associated with acute respiratory failure, anaemia, systemic inflammatory response syndrome, circulatory shock and major surgery. The model showed an area under the curve (AUC) of 0.907 [95% confidence interval (CI) 0.902–0.908), a sensitivity of 82.7 (95% CI 80.7–84.6) and a specificity of 84.2 (95% CI 83.9–84.6) to predict HA-AKI, with an adequate goodness-of-fit for all risk categories (χ² = 6.02, P = 0.64). In the validation set, the prevalence of AKI was 3.2%. The model showed an AUC of 0.905 (95% CI 0.904–0.910), a sensitivity of 81.2 (95% CI 79.2–83.1) and a specificity of 82.5 (95% CI 82.2–83) to predict HA-AKI and had an adequate goodness-of-fit for all risk categories (χ² = 4.2, P = 0.83). An online tool (predaki.amalfianalytics.com) is available to calculate the risk of AKI in other hospital environments.

Conclusions

By using electronic health data records, our study provides a model that can be used in clinical practice to obtain an accurate dynamic and updated assessment of the individual risk of HA-AKI during the hospital admission period in non-critically ill patients.

The potential for artificial intelligence to predict clinical outcomes in patients who have acquired acute kidney injury during the perioperative period

Acute kidney injury (AKI) is a common medical problem in hospitalised patients worldwide that may result in negative physiological, social and economic consequences. Amongst patients admitted to ICU with AKI, over 40% have had either elective or emergency surgery prior to admission. Predicting outcomes after AKI is difficult and the decision on whom to initiate RRT with a goal of renal recovery or predict a long-term survival benefit still poses a challenge for acute care physicians. With the increasing use of electronic healthcare records, artificial intelligence may allow postoperative AKI prognostication and aid clinical management. Patients will benefit if the data can be readily accessed andregulatory, ethical and human factors challenges can be overcome.

External validation of the Madrid Acute Kidney Injury Prediction Score

Background

The Madrid Acute Kidney Injury Prediction Score (MAKIPS) is a recently described tool capable of performing automatic calculations of the risk of hospital-acquired acute kidney injury (HA-AKI) using data from from electronic clinical records that could be easily implemented in clinical practice. However, to date, it has not been externally validated. The aim of our study was to perform an external validation of the MAKIPS in a hospital with different characteristics and variable case mix.

Methods

This external validation cohort study of the MAKIPS was conducted in patients admitted to a single tertiary hospital between April 2018 and September 2019. Performance was assessed by discrimination using the area under the receiver operating characteristics curve and calibration plots.

Results

A total of 5.3% of the external validation cohort had HA-AKI. When compared with the MAKIPS cohort, the validation cohort showed a higher percentage of men as well as a higher prevalence of diabetes, hypertension, cardiovascular disease, cerebrovascular disease, anaemia, congestive heart failure, chronic pulmonary disease, connective tissue diseases and renal disease, whereas the prevalence of peptic ulcer disease, liver disease, malignancy, metastatic solid tumours and acquired immune deficiency syndrome was significantly lower. In the validation cohort, the MAKIPS showed an area under the curve of 0.798 (95% confidence interval 0.788–0.809). Calibration plots showed that there was a tendency for the MAKIPS to overestimate the risk of HA-AKI at probability rates ˂0.19 and to underestimate at probability rates between 0.22 and 0.67.

Conclusions

The MAKIPS can be a useful tool, using data that are easily obtainable from electronic records, to predict the risk of HA-AKI in hospitals with different case mix characteristics.

Balancing the Needs of Acute and Maintenance Dialysis Patients during the COVID-19 Pandemic: A Proposed Ethical Framework for Dialysis Allocation

The COVID-19 pandemic continues to strain health care systems and drive shortages in medical supplies and equipment around the world. Resource allocation in times of scarcity requires transparent, ethical frameworks to optimize decision making and reduce health care worker and patient distress. The complexity of allocating dialysis resources for both patients receiving acute and maintenance dialysis has not previously been addressed. Using a rapid, collaborative, and iterative process, BC Renal, a provincial network in Canada, engaged patients, doctors, ethicists, administrators, and nurses to develop a framework for addressing system capacity, communication challenges, and allocation decisions. The guiding ethical principles that underpin this framework are (1) maximizing benefits, (2) treating people fairly, (3) prioritizing the worst-off individuals, and (4) procedural justice. Algorithms to support resource allocation and triage of patients were tested using simulations, and the final framework was reviewed and endorsed by members of the provincial nephrology community. The unique aspects of this allocation framework are the consideration of two diverse patient groups who require dialysis (acute and maintenance), and the application of two allocation criteria (urgency and prognosis) to each group in a sequential matrix. We acknowledge the context of the Canadian health care system, and a universal payer in which this framework was developed. The intention is to promote fair decision making and to maintain an equitable reallocation of limited resources for a complex problem during a pandemic.

Using Administrative Data to Assess the Risk of Permanent Work Disability: A Cohort Study

Purpose Unmet rehabilitation needs are common. We therefore developed a risk score using administrative data to assess the risk of permanent work disability. Such a score may support the identification of individuals with a high likelihood of receiving a disability pension. Methods Our sample was a random and stratified 1% sample of individuals aged 18-65 years paying pension contributions. From administrative records, we extracted sociodemographic data and data about employment and welfare benefits covering 2010-2012. Our outcome was a pension due to work disability that was requested between January 2013 and December 2017. We developed a comprehensive logistic regression model and used the model estimates to determine the risk score. Results We included 352,140 individuals and counted 6,360 (1.8%) disability pensions during the 5-year follow-up. The area under the receiver operating curve was 0.839 (95% CI 0.834 to 0.844) for the continuous risk score. Using a threshold of ≥ 50 points (20.2% of all individuals), we correctly classified 80.6% of all individuals (sensitivity: 71.5%; specificity: 80.8%). Using ≥ 60 points (9.9% of all individuals), we correctly classified 90.3% (sensitivity: 54.9%; specificity: 91.0%). Individuals with 50 to < 60 points had a five times higher risk of a disability pension compared to individuals with low scores, individuals with ≥ 60 points a 17 times higher risk. Conclusions The risk score offers an opportunity to screen for people with a high risk of permanent work disability.

Timing of continuous renal replacement therapy in severe acute kidney injury patients with fluid overload: A retrospective cohort study

PURPOSE

We aimed to evaluate the association of early versus late initiation of Continuous renal replacement therapy (CRRT) with mortality in patients with fluid overload.

METHODS

This was a retrospective cohort study of patients with fluid overload (FO) treated with CRRT due to severe acute kidney injury (AKI) between January 2015 and December 2017 in a mixed medical intensive care unit of a teaching hospital in Beijing, China. Patients were divided into early (≤15 h) and late (>15 h) groups based on the median time from ICU admission to CRRT initiation. The primary outcome was all-cause mortality at day 60. Multivariable Cox model analysis was used for analysis.

RESULTS

The study patients were male predominant (84/150) with a mean age of 64.8 ± 16.7 years. The median FO value before CRRT initiation was 10.1% [6.2-16.1%]. The 60-day mortality rates in the early vs the late CRRT groups were 53.9% and 73%, respectively. On multivariable Cox modelling, the late initiation of CRRT was independently associated with an increased risk of death at 60 days (HR 1.75, 95% CI 1.11-2.74, p = 0.015).

CONCLUSIONS

Early initiation of CRRT was independently associated with survival benefits in severe AKI patients with fluid overload.

A prediction and interpretation framework of acute kidney injury in critical care

Acute kidney injury (AKI) is a common clinical condition with high mortality and resource consumption. Early identification of high-risk patients to achieve an appropriate allocation of limited clinical resources and timely interventions is of significant importance, which has attracted substantial research to develop prediction models for AKI risk stratification. However, most available AKI prediction models have moderate performance and lack of interpretability, which limits their applicability in supporting care intervention. In this paper, a machine learning-based framework for AKI prediction and interpretation in critical care is presented. First, an ensemble model is developed to predict a patient's risk of AKI within 72 h of admission to the intensive care units. Next, the model is interpreted both globally and locally. For the global interpretation, the important predictors are pinpointed and the detailed relationships between AKI risk and these predictors are illustrated. For the local interpretation, patient-specific analysis is presented to provide a visualized explanation for each individual prediction. Experimental results show that such a prediction and interpretation framework can lead to good prediction and interpretation performance, which has the potential to provide effective clinical decision support.

Prediction models for acute kidney injury in patients with gastrointestinal cancers: a real-world study based on Bayesian networks

Background

This study attempts to establish a Bayesian networks (BNs) based model for inferring the risk of AKI in gastrointestinal cancer (GI) patients, and to compare its predictive capacity with other machine learning (ML) models.

Methods

From 1 October 2014 to 30 September 2015, we recruited 6495 inpatients with GI cancers in a tertiary hospital in eastern China. Data on demographics, clinical and laboratory indicators were retrospectively extracted from the electronic medical record system. Predictors of AKI were selected in gLASSO regression, and further incorporated into BNs analysis.

Results

The incidences of AKI in patients with esophagus, stomach, and intestine cancer were 20.5%, 13.9%, and 12.5%, respectively. Through gLASSO, 11 predictors were screened out, including diabetes, cancer category, anti-tumor treatment, ALT, serum creatinine, estimated glomerular filtration rate (eGFR), serum uric acid (SUA), hypoalbuminemia, anemia, abnormal sodium, and potassium. BNs model revealed that cancer category, treatment, eGFR, and hypoalbuminemia had direct connections with AKI. Diabetes and SUA were indirectly linked to AKI through eGFR, and anemia created connections with AKI through affecting album level. Compared with other ML models, BNs model maintained a higher AUC value in both the internal and external validation (AUC: 0.823/0.790).

Conclusion

BNs model not only delineates the qualitative and quantitative relationship between AKI and its associated factors but shows the more robust generalizability in AKI prediction.

Negative Regulation of Tec Kinase Alleviates LPS-Induced Acute Kidney Injury in Mice via theTLR4/NF-κB Signaling Pathway

Tec kinase is an important mediator in inflammatory immune response that enhances the activity of neutrophils and macrophages. However, information on its function in lipopolysaccharide- (LPS-) induced acute kidney injury (AKI) is limited. This study is aimed at determining whether Tec kinase was a regulator in AKI. An AKI model in mice was successfully established using intraperitoneal LPS. Results showed that the serum levels of creatinine (Cr), blood urea nitrogen (BUN), and cystatin-C (Cys-C) increased after intraperitoneal LPS injection. Renal tissue sustained significantly severe injury as measured by pathological scores. Pretreatment with LFM-A13 improved the function of the kidney in mice and decreased the renal injury score. Enzyme-linked immunosorbent assay showed that LFM-A13 significantly reduced the release of IL-1β and TNF-α in mice exposed to LPS. LFM-A13 can evidently abrogate the expression of Tec protein, MyD88, TLR4, NF-κB p65, and Tec's phosphorylated protein as determined by Western blot. Immunohistochemistry analysis revealed that LFM-A13 markedly downregulated the expression of Tec kinase in renal tubular epithelial cells. In vitro, Tec kinase protein was expressed highly in NRK-52E cells after LPS exposure. Tec-siRNA also decreased IL-1β and TNF-α production and obviously abolished phospho-p65 and phospho-IκBα expression in NRK-52E cell stimulated by LPS; however, Tec-siRNA increased the IκBα level. Altogether, these data suggested that Tec kinase can be a modulating protein in AKI through TLR4/NF-κB activation.

A novel machine learning algorithm, Bayesian networks model, to predict the high-risk patients with cardiac surgery-associated acute kidney injury

BACKGROUND

Cardiac surgery-associated acute kidney injury (CSA-AKI) is a well-recognized complication with an ominous outcome.

HYPOTHESIS

Bayesian networks (BNs) not only can reveal the complex interrelationships between predictors and CSA-AKI, but predict the individual risk of CSA-AKI occurrence.

METHODS

During 2013 and 2015, we recruited 5533 eligible participants who underwent cardiac surgery from a tertiary hospital in eastern China. Data on demographics, clinical and laboratory information were prospectively recorded in the electronic medical system and analyzed by gLASSO-logistic regression and BNs.

RESULTS

The incidences of CSA-AKI and severe CSA-AKI were 37.5% and 11.1%. BNs model revealed that gender, left ventricular ejection fractions (LVEF), serum creatinine (SCr), serum uric acid (SUA), platelet, and aortic cross-clamp time (ACCT) were found as the parent nodes of CSA-AKI, while ultrafiltration volume and postoperative central venous pressure (CVP) were connected with CSA-AKI as children nodes. In the severe CSA-AKI model, age, proteinuria, and SUA were directly linked to severe AKI; the new nodes of NYHA grade and direct bilirubin created relationships with severe AKI through was related to LVEF, surgery types, and SCr level. The internal AUCs for predicting CSA-AKI and severe AKI were 0.755 and 0.845, which remained 0.736 and 0.816 in the external validation. Given the known variables, the risk for CSA-AKI can be inferred at individual levels based on the established BNs model and prior information.

CONCLUSION

BNs model has a high accuracy, good interpretability, and strong generalizability in predicting CSA-AKI. It facilitates physicians to identify high-risk patients and implement protective strategies to improve the prognosis.

Stigma, biomarkers, and algorithmic bias: recommendations for precision behavioral health with artificial intelligence

Effective implementation of artificial intelligence in behavioral healthcare delivery depends on overcoming challenges that are pronounced in this domain. Self and social stigma contribute to under-reported symptoms, and under-coding worsens ascertainment. Health disparities contribute to algorithmic bias. Lack of reliable biological and clinical markers hinders model development, and model explainability challenges impede trust among users. In this perspective, we describe these challenges and discuss design and implementation recommendations to overcome them in intelligent systems for behavioral and mental health.

Predicting mortality among critically ill patients with acute kidney injury treated with renal replacement therapy: Development and validation of new prediction models

PURPOSE

Severe acute kidney injury (AKI) is associated with a significant risk of mortality and persistent renal replacement therapy (RRT) dependence. The objective of this study was to develop prediction models for mortality at 90-day and 1-year following RRT initiation in critically ill patients with AKI.

METHODS

All patients who commenced RRT in the intensive care unit for AKI at a tertiary care hospital between 2007 and 2014 constituted the development cohort. We evaluated the external validity of our mortality models using data from the multicentre OPTIMAL-AKI study.

RESULTS

The development cohort consisted of 594 patients, of whom 320(54%) died and 40 (15% of surviving patients) remained RRT-dependent at 90-day Eleven variables were included in the model to predict 90-day mortality (AUC:0.79, 95%CI:0.76-0.82). The performance of the 90-day mortality model declined upon validation in the OPTIMAL-AKI cohort (AUC:0.61, 95%CI:0.54-0.69) and showed modest calibration. Similar results were obtained for mortality model at 1-year.

CONCLUSIONS

Routinely collected variables at the time of RRT initiation have limited ability to predict mortality in critically ill patients with AKI who commence RRT.

Acute kidney injury: a risk scoring system for general surgical patients

This article describes the development of a scoring system for general surgical patients to highlight those at greater risk of developing acute kidney injury (AKI). Following a search of the literature on current practice, a list of common variables was composed. Hospital Episode Statistics (HES) data from two random hospital trusts was used. With the help of a risk analysis system (CRAB Medical module, CRAB Clinical Informatics Ltd) it was possible to examine the relationship between potential risk factors and the incidence of AKI. Using Analyse-it for Excel a binary logistic model was created, which led to the development of a logistic regression equation and consequently a scoring system. The sensitivity and specificity of the model was tested using the receiver operating characteristic (ROC) curve. There was good correlation across the whole risk spectrum with an area under ROC curve of 0.806 (95% confidence intervals 0.787-0.825). The scoring system was developed into an admission checklist for general surgical patients to highlight a patient's risk of developing AKI. In a ward setting a checklist that immediately assesses the patient and produces a rapid indication as to whether the patient is at high risk or low risk would seem to be the ideal tool.

Developing and testing models to predict mortality in the general population

We have previously proposed an approach using information collected from published reports to generate prediction models. The goal of this project was to validate this technique to develop and test various prediction models. A risk indicator (R) is calculated as a linear combination of the hazard ratios for the following predictors: age, male gender, diabetes, albuminuria, and either CKD, CVD or both. We developed a linear and two exponential expressions to predict the probability of the outcome of 2-year mortality and compared to actual outcome in the target dataset from NHANES. The risk indicator demonstrated good performance with area under ROC curve of 0.84. The linear and two exponential expressions generated similar predictions in the lower categories of risk indicator (R ≤ 6). However, in the groups with higher R value, the linear expression tends to predict lower, and the exponential expressions higher, probabilities than the observed outcome. A Combined model which averaged the linear and logistic expressions was shown to approximate the actual outcome data the best. A simple technique (named Woodpecker™) allows derivation functional prediction models and risk stratification tools from reports of clinical outcome studies and their application to new populations by using only summary statistics of the new population.

Does the SORG Algorithm Predict 5-year Survival in Patients with Chondrosarcoma? An External Validation

BACKGROUND

We developed a machine learning algorithm to predict the survival of patients with chondrosarcoma. The algorithm demonstrated excellent discrimination and calibration on internal validation in a derivation cohort based on data from the Surveillance, Epidemiology, and End Results (SEER) registry. However, the algorithm has not been validated in an independent external dataset.

QUESTIONS/PURPOSES

Does the Skeletal Oncology Research Group (SORG) algorithm accurately predict 5-year survival in an independent patient population surgically treated for chondrosarcoma?

METHODS

The SORG algorithm was developed using the SEER registry, which contains demographic data, tumor characteristics, treatment, and outcome values; and includes approximately 30% of the cancer patients in the United States. The SEER registry was ideal for creating the derivation cohort, and consequently the SORG algorithm, because of the high number of eligible patients and the availability of most (explanatory) variables of interest. Between 1992 to 2013, 326 patients were treated surgically for extracranial chondrosarcoma of the bone at two tertiary care referral centers. Of those, 179 were accounted for at a minimum of 5 years after diagnosis in a clinical note at one of the two institutions, unless they died earlier, and were included in the validation cohort. In all, 147 (45%) did not meet the minimum 5 years of followup at the institution and were not included in the validation of the SORG algorithm. The outcome (survival at 5 years) was checked for all 326 patients in the Social Security death index and were included in the supplemental validation cohort, to also ascertain validity for patients with less than 5 years of institutional followup. Variables used in the SORG algorithm to predict 5-year survival including sex, age, histologic subtype, tumor grade, tumor size, tumor extension, and tumor location were collected manually from medical records. The tumor characteristics were collected from the postoperative musculoskeletal pathology report. Predicted probabilities of 5-year survival were calculated for each patient in the validation cohort using the SORG algorithm, followed by an assessment of performance using the same metrics as used for internal validation, namely: discrimination, calibration, and overall performance. Discrimination was calculated using the concordance statistic (or the area under the Receiver Operating Characteristic (ROC) curve) to determine how well the algorithm discriminates between the outcome, which ranges from 0.5 (no better than a coin-toss) to 1.0 (perfect discrimination). Calibration was assessed using the calibration slope and intercept from a calibration plot to measure the agreement between predicted and observed outcomes. A perfect calibration plot should show a 45° upwards line. Overall performance was determined using the Brier score, ranging from 0 (excellent prediction) to 1 (worst prediction). The Brier score was compared with the null-model Brier score, which showed the performance of a model that ignored all the covariates. A Brier score lower than the null model Brier score indicated greater performance of the algorithm. For the external validation an F1-score was added to measure the overall accuracy of the algorithm, which ranges between 0 (total failure of an algorithm) and 1 (perfect algorithm).The 5-year survival was lower in the validation cohort than it was in the derivation cohort from SEER (61.5% [110 of 179] versus 76% [1131 of 1544] ; p < 0.001). This difference was driven by higher proportion of dedifferentiated chondrosarcoma in the institutional population than in the derivation cohort (27% [49 of 179] versus 9% [131 of 1544]; p < 0.001). Patients in the validation cohort also had larger tumor sizes, higher grades, and nonextremity tumor locations than did those in the derivation cohort. These differences between the study groups emphasize that the external validation is performed not only in a different patient cohort, but also in terms of disease characteristics. Five-year survival was not different for both patient groups between subpopulations of patients with conventional chondrosarcomas and those with dedifferentiated chondrosarcomas.

RESULTS

The concordance statistic for the validation cohort was 0.87 (95% CI, 0.80-0.91). Evaluation of the algorithm's calibration in the institutional population resulted in a calibration slope of 0.97 (95% CI, 0.68-1.3) and calibration intercept of -0.58 (95% CI, -0.20 to -0.97). Finally, on overall performance, the algorithm had a Brier score of 0.152 compared with a null-model Brier score of 0.237 for a high level of overall performance. The F1-score was 0.836. For the supplementary validation in the total of 326 patients, the SORG algorithm had a validation of 0.89 (95% CI, 0.85-0.93). The calibration slope was 1.13 (95% CI, 0.87-1.39) and the calibration intercept was -0.26 (95% CI, -0.57 to 0.06). The Brier score was 0.11, with a null-model Brier score of 0.19. The F1-score was 0.901.

CONCLUSIONS

On external validation, the SORG algorithm retained good discriminative ability and overall performance but overestimated 5-year survival in patients surgically treated for chondrosarcoma. This internet-based tool can help guide patient counseling and shared decision making.

LEVEL OF EVIDENCE

Level III, prognostic study.

Development and validation of a prediction model for the probability of responding to placebo in antidepressant trials: a pooled analysis of individual patient data

BACKGROUND

Identifying potential placebo responders among apparent drug responders is critical to dissect drug-specific and nonspecific effects in depression.

OBJECTIVE

This project aimed to develop and test a prediction model for the probability of responding to placebo in antidepressant trials. Such a model will allow us to estimate the probability of placebo response among drug responders in antidepressants trials.

METHODS

We identified all placebo-controlled, double-blind randomised controlled trials (RCTs) of second generation antidepressants for major depressive disorder conducted in Japan and requested their individual patient data (IPD) to pharmaceutical companies. We obtained IPD (n=1493) from four phase II/III RCTs comparing mirtazapine, escitalopram, duloxetine, paroxetine and placebo. Out of 1493 participants in the four clinical trials, 440 participants allocated to placebo were included in the analyses. Our primary outcome was response, defined as 50% or greater reduction on Hamilton Rating Scale for Depression at study endpoint. We used multivariable logistic regression to develop a prediction model. All available candidate of predictor variables were tested through a backward variable selection and covariates were selected for the prediction model. The performance of the model was assessed by using Hosmer-Lemeshow test for calibration and the area under the ROC curve for discrimination.

FINDINGS

Placebo response rates differed between 31% and 59% (grand average: 43%) among four trials. Four variables were selected from all candidate variables and included in the final model: age at onset, age at baseline, bodily symptoms, and study-level difference. The final model performed satisfactorily in terms of calibration (Hosmer-Lemeshow p=0.92) and discrimination (the area under the ROC curve (AUC): 0.70).

CONCLUSIONS

Our model is expected to help researchers discriminate individuals who are more likely to respond to placebo from those who are less likely so.

CLINICAL IMPLICATIONS

A larger sample and more precise individual participant information should be collected for better performance. Examination of external validity in independent datasets is warranted.

TRIAL REGISTRATION NUMBER

CRD42017055912.

Causal risk factor discovery for severe acute kidney injury using electronic health records

BACKGROUND

Acute kidney injury (AKI), characterized by abrupt deterioration of renal function, is a common clinical event among hospitalized patients and it is associated with high morbidity and mortality. AKI is defined in three stages with stage-3 being the most severe phase which is irreversible. It is important to effectively discover the true risk factors in order to identify high-risk AKI patients and allow better targeting of tailored interventions. However, Stage-3 AKI patients are very rare (only 0.2% of AKI patients) with a large scale of features available in EHR (1917 potential risk features), yielding a scenario unfeasible for any correlation-based feature selection or modeling method. This study aims to discover the key factors and improve the detection of Stage-3 AKI.

METHODS

A causal discovery method (McDSL) is adopted for causal discovery to infer true causal relationship between information buried in EHR (such as medication, diagnosis, laboratory tests, comorbidities and etc.) and Stage-3 AKI risk. The research approach comprised two major phases: data collection, and causal discovery. The first phase is propose to collect the data from HER (includes 358 encounters and 891 risk factors). Finally, McDSL is employed to discover the causal risk factors of Stage-3 AKI, and five well-known machine learning models are built for predicting Stage-3 AKI with 10-fold cross-validation (predictive accuracy were measured by AUC, precision, recall and F-score).

RESULTS

McDSL is useful for further research of EHR. It is able to discover four causal features, all selected features are medications that are modifiable. The latest research of machine learning is employed to compare the performance of prediction, and the experimental result has verified the selected features are pivotal.

CONCLUSIONS

The features selected by McDSL, which enable us to achieve significant dimension reduction without sacrificing prediction accuracy, suggesting potential clinical use such as helping physicians develop better prevention and treatment strategies.

Definition, Management, and Outcomes of Acute Kidney Injury: An International Survey of Nephrologists

Background

Acute kidney injury (AKI) is a complex disease burdened by uncertainties of definition, management strategies, and prognosis. This study explores the relationship between demographic characteristics of nephrologists and their perceptions about the definition, management, and follow-up of AKI.

Methods

We developed a Web-based survey, the International Survey on Acute Kidney Injury (ISAKI), consisting of 29 items in 4 categories: (1) demographic and practice characteristics, (2) definition of AKI, (3) management of renal replacement therapy (RRT) in AKI, and (4) sequelae of AKI. A multivariable stepwise logistic regression model was used to examine relationships between the dependent variables and the demographic characteristics of the respondents.

Results

Responses from 743 nephrologists from 90 countries were analyzed. The majority (60%) of respondents reported using RIFLE and/or AKIN criteria regularly to define AKI, although US nephrologists were less likely to do so (OR: 0.58; 95% CI: 0.42-0.85). The most common initial RRT modality was intermittent hemodialysis (63.5%), followed by continuous RRT (23.8%). Faculty affiliation was associated with a higher likelihood of using a dialysis schedule of ≥4 times a week (OR: 1.75; 95% CI: 1.20-2.55). The respondents believed that a single episode of AKI increases the likelihood of development of chronic kidney disease (CKD) (55%), subsequent AKI (36%), and rapid progression of preexisting CKD (87%). US nephrologists were less likely to recommend follow-up after resolution of AKI (OR: 0.15; 95% CI: 0.07-0.33).

Conclusions

Our findings highlight the need for a widely accepted consensus definition of AKI, a uniform approach to management, and improved follow-up after resolution of AKI episodes.