The ethical matrix as a method for involving people living with disease and the wider public (PPI) in near-term artificial intelligence research

Using artificial intelligence to predict patient outcomes from patient-reported outcome measures: a scoping review

PURPOSE

This scoping review aims to identify and summarise artificial intelligence (AI) methods applied to patient-reported outcome measures (PROMs) for prediction of patient outcomes, such as survival, quality of life, or treatment decisions.

INTRODUCTION

AI models have been successfully applied to predict outcomes for patients using mainly clinically focused data. However, systematic guidance for utilising AI and PROMs for patient outcome predictions is lacking. This leads to inconsistency of model development and evaluation, limited practical implications, and poor translation to clinical practice.

MATERIALS AND METHODS

This review was conducted across Web of Science, IEEE Xplore, ACM, Digital Library, Cochrane Central Register of Controlled Trials, Medline and Embase databases. Adapted search terms identified published research using AI models with patient-reported data for outcome predictions. Papers using PROMs data as input variables in AI models for prediction of patient outcomes were included.

RESULTS

Three thousand and seventy-seven records were screened, 94 of which were included in the analysis. AI models applied to PROMs data for outcome predictions are most commonly used in orthopaedics and oncology. Poor reporting of model hyperparameters and inconsistent techniques of handling class imbalance and missingness in data were found. The absence of external model validation, participants' ethnicity information and stakeholders involvement was common.

CONCLUSION

The results highlight inconsistencies in conducting and reporting of AI research involving PROMs in patients' outcomes predictions, which reduces the reproducibility of the studies. Recommendations for external validation and stakeholders' involvement are given to increase the opportunities for applying AI models in clinical practice.

iSPAN: Explainable prediction of outcomes post thrombectomy with Machine Learning

PURPOSE

This study aimed to develop and evaluate a machine learning model and a novel clinical score for predicting outcomes in stroke patients undergoing endovascular thrombectomy.

MATERIALS AND METHODS

This retrospective study included all patients aged over 18 years with an anterior circulation stroke treated at a thrombectomy centre from 2010 to 2020 with external validation. The primary outcome was day 90 mRS ≥3. Existing clinical scores (SPAN and PRE) and Machine Learning (ML) models were compared. A novel clinical score (iSPAN) was derived by adding an optimised weighting of the most important ML features to the SPAN.

RESULTS

812 patients were initially included (397 female, average age 73), 63 for external validation. The best performing clinical score and ML model were SPAN and XGB (sensitivity, specificity and accuracy 0.290, 0.967, 0.628 and 0.693, 0.783, 0.738 respectively). A significant difference was found overall and our XGB model was more accurate than SPAN (p < 0.0018). The most important features were Age, mTICI and total number of passes. The addition of 11 points for mTICI of ≤2B and 3 points for ≥3 passes to the SPAN achieved the best accuracy and was used to create the iSPAN. iSPAN was not significantly less accurate than our XGB model (p > 0.5). In the external validation set, iSPAN and SPAN achieved sensitivity, specificity, and accuracy of (0.735, 0.862, 0.79) and (0.471, 0.897, 0.67) respectively.

CONCLUSION

iSPAN incorporates machine-derived features to achieve better predictions compared to existing clinical scores. It is not inferior to our XGB model and is externally generalisable.

A Radiomic "Warning Sign" of Progression on Brain MRI in Individuals with MS

BACKGROUND AND PURPOSE

MS is a chronic progressive, idiopathic, demyelinating disorder whose diagnosis is contingent on the interpretation of MR imaging. New MR imaging lesions are an early biomarker of disease progression. We aimed to evaluate a machine learning model based on radiomics features in predicting progression on MR imaging of the brain in individuals with MS.

MATERIALS AND METHODS

This retrospective cohort study with external validation on open-access data obtained full ethics approval. Longitudinal MR imaging data for patients with MS were collected and processed for machine learning. Radiomics features were extracted at the future location of a new lesion in the patients' prior MR imaging ("prelesion"). Additionally, "control" samples were obtained from the normal-appearing white matter for each participant. Machine learning models for binary classification were trained and tested and then evaluated the external data of the model.

RESULTS

The total number of participants was 167. Of the 147 in the training/test set, 102 were women and 45 were men. The average age was 42 (range, 21-74 years). The best-performing radiomics-based model was XGBoost, with accuracy, precision, recall, and F1-score of 0.91, 0.91, 0.91, and 0.91 on the test set, and 0.74, 0.74, 0.74, and 0.70 on the external validation set. The 5 most important radiomics features to the XGBoost model were associated with the overall heterogeneity and low gray-level emphasis of the segmented regions. Probability maps were produced to illustrate potential future clinical applications.

CONCLUSIONS

Our machine learning model based on radiomics features successfully differentiated prelesions from normal-appearing white matter. This outcome suggests that radiomics features from normal-appearing white matter could serve as an imaging biomarker for progression of MS on MR imaging.