The quality of research with real-world evidence

A machine learning model for predicting abnormal liver function induced by a Chinese herbal medicine preparation (Zhengqing Fengtongning) in patients with rheumatoid arthritis based on real-world study

OBJECTIVE

Rheumatoid arthritis (RA) is a systemic autoimmune disease that affects the small joints of the whole body and degrades the patients' quality of life. Zhengqing Fengtongning (ZF) is a traditional Chinese medicine preparation used to treat RA. ZF may cause liver injury. In this study, we aimed to develop a prediction model for abnormal liver function caused by ZF.

METHODS

This retrospective study collected data from multiple centers from January 2018 to April 2023. Abnormal liver function was set as the target variable according to the alanine transaminase (ALT) level. Features were screened through univariate analysis and sequential forward selection for modeling. Ten machine learning and deep learning models were compared to find the model that most effectively predicted liver function from the available data.

RESULTS

This study included 1,913 eligible patients. The LightGBM model exhibited the best performance (accuracy = 0.96) out of the 10 learning models. The predictive metrics of the LightGBM model were as follows: precision = 0.99, recall rate = 0.97, F1_score = 0.98, area under the curve (AUC) = 0.98, sensitivity = 0.97 and specificity = 0.85 for predicting ALT < 40 U/L; precision = 0.60, recall rate = 0.83, F1_score = 0.70, AUC = 0.98, sensitivity = 0.83 and specificity = 0.97 for predicting 40 ≤ ALT < 80 U/L; and precision = 0.83, recall rate = 0.63, F1_score = 0.71, AUC = 0.97, sensitivity = 0.63 and specificity = 1.00 for predicting ALT ≥ 80 U/L. ZF-induced abnormal liver function was found to be associated with high total cholesterol and triglyceride levels, the combination of TNF-α inhibitors, JAK inhibitors, methotrexate + nonsteroidal anti-inflammatory drugs, leflunomide, smoking, older age, and females in middle-age (45-65 years old).

CONCLUSION

This study developed a model for predicting ZF-induced abnormal liver function, which may help improve the safety of integrated administration of ZF and Western medicine. Please cite this article as: Yu Z, Kou F, Gao Y, Lyu CM, Gao F, Wei H. A machine learning model for predicting abnormal liver function induced by a Chinese herbal medicine preparation (Zhengqing Fengtongning) in patients with rheumatoid arthritis based on real-world study. J Integr Med. 2025; 23(1): 25-35.

Effectiveness of ulinastatin in critical care patients in real world: a retrospective multi-center study

OBJECTIVES

Ulinastatin has been applied in various diseases associated with inflammation, but its effectiveness lacks real-world evidence. We aimed to evaluate the effectiveness of ulinastatin based on a real-world database in the intensive care unit (ICU) patients.

METHODS

This was a retrospective cohort study. ICU patient data from multi-centers in China were collected. Propensity score matching (PSM) was applied. The effectiveness of ulinastatin was evaluated by mortality, length of stay in the ICU and mechanical ventilation duration. Kaplan-Meier method was used to estimate the hazard ratio and plot the survival curve.

RESULTS

A total of 2036 patients were analyzed after PSM. Mortality was significantly lower in the ulinastatin group than in the non-ulinastatin group (hazard ratio for death: 0.77; 95% confidence interval: 0.62-0.96; p = 0.018). Ulinastatin significantly reduced mortality at 28 days (10.0% vs. 13.6%), 60 days (13.9% vs. 18.2%) and 90 days (14.7% vs. 18.5%), length of stay in the ICU (median 8.0 d vs. 13.0 d), and mechanical ventilation duration (median 24.0 h vs. 25.0 h; p < 0.05).

CONCLUSIONS

Ulinastatin was beneficial for patients in the ICU, mainly by reducing mortality, length of ICU stay, and mechanical ventilation duration. This study provides evidence of the clinical effectiveness of ulinastatin.

Personalized venlafaxine dose prediction using artificial intelligence technology: a retrospective analysis based on real-world data

BACKGROUND

Venlafaxine dose regimens vary considerably between individuals, requiring personalized dosing.

AIM

This study aimed to identify dose-related influencing factors of venlafaxine through real-world data analysis and to construct a personalized dose model using advanced artificial intelligence techniques.

METHOD

We conducted a retrospective study on patients with depression treated with venlafaxine. Significant variables were selected through a univariate analysis. Subsequently, the predictive performance of seven models (XGBoost, LightGBM, CatBoost, GBDT, ANN, TabNet, and DT) was compared. The algorithm that demonstrated optimal performance was chosen to establish the dose prediction model. Model validation used confusion matrices and ROC analysis. Additionally, a dose subgroup analysis was conducted.

RESULTS

A total of 298 patients were included. TabNet was selected to establish the venlafaxine dose prediction model, which exhibited the highest performance with an accuracy of 0.80. The analysis identified seven crucial variables correlated with venlafaxine daily dose, including blood venlafaxine concentration, total protein, lymphocytes, age, globulin, cholinesterase, and blood platelet count. The area under the curve (AUC) for predicting venlafaxine doses of 75 mg, 150 mg, and 225 mg were 0.90, 0.85, and 0.90, respectively.

CONCLUSION

We successfully developed a TabNet model to predict venlafaxine doses using real-world data. This model demonstrated substantial predictive accuracy, offering a personalized dosing regimen for venlafaxine. These findings provide valuable guidance for the clinical use of the drug.

Developing a machine learning model for predicting venlafaxine active moiety concentration: a retrospective study using real-world evidence

BACKGROUND

Venlafaxine is frequently prescribed for patients with depression. To control the concentration of venlafaxine within the therapeutic window for the best treatment effect, a model to predict venlafaxine concentration is necessary.

AIM

Our objective was to develop a prediction model for venlafaxine concentration using real-world evidence based on machine learning and deep learning techniques.

METHOD

Patients who underwent venlafaxine treatment between November 2019 and August 2022 were included in the study. Important variables affecting venlafaxine concentration were identified using a combination of univariate analysis, sequential forward selection, and machine learning techniques. Predictive performance of nine machine learning and deep learning algorithms were assessed, and the one with the optimal performance was selected for modeling. The final model was interpreted using SHapley Additive exPlanations.

RESULTS

A total of 330 eligible patients were included. Five influential variables that affect venlafaxine concentration were venlafaxine daily dose, sex, age, hyperlipidemia, and adenosine deaminase. The venlafaxine concentration prediction model was developed using the eXtreme Gradient Boosting algorithm (R2 = 0.65, mean absolute error = 77.92, root mean square error = 93.58). In the testing cohort, the accuracy of the predicted concentration within ± 30% of the actual concentration was 73.49%. In the subgroup analysis, the prediction accuracy was 69.39% within the recommended therapeutic range of venlafaxine concentration within ± 30% of the actual value.

CONCLUSION

The XGBoost model for predicting blood concentration of venlafaxine using real-world evidence was developed, guiding the adjustment of regimen in clinical practice.

Predicting Lapatinib Dose Regimen Using Machine Learning and Deep Learning Techniques Based on a Real-World Study

Lapatinib is used for the treatment of metastatic HER2(+) breast cancer. We aim to establish a prediction model for lapatinib dose using machine learning and deep learning techniques based on a real-world study. There were 149 breast cancer patients enrolled from July 2016 to June 2017 at Fudan University Shanghai Cancer Center. The sequential forward selection algorithm based on random forest was applied for variable selection. Twelve machine learning and deep learning algorithms were compared in terms of their predictive abilities (logistic regression, SVM, random forest, Adaboost, XGBoost, GBDT, LightGBM, CatBoost, TabNet, ANN, Super TML, and Wide&Deep). As a result, TabNet was chosen to construct the prediction model with the best performance (accuracy = 0.82 and AUC = 0.83). Afterward, four variables that strongly correlated with lapatinib dose were ranked via importance score as follows: treatment protocols, weight, number of chemotherapy treatments, and number of metastases. Finally, the confusion matrix was used to validate the model for a dose regimen of 1,250 mg lapatinib (precision = 81% and recall = 95%), and for a dose regimen of 1,000 mg lapatinib (precision = 87% and recall = 64%). To conclude, we established a deep learning model to predict lapatinib dose based on important influencing variables selected from real-world evidence, to achieve an optimal individualized dose regimen with good predictive performance.