Artificial intelligence applied to medicine: There is an "elephant in the room"

Inter-institutional variability of knowledge-based plan prediction of left whole breast irradiation

PURPOSE

Within a multi-institutional project, we aimed to assess the transferability of knowledge-based (KB) plan prediction models in the case of whole breast irradiation (WBI) for left-side breast irradiation with tangential fields (TF).

METHODS

Eight institutions set KB models, following previously shared common criteria. Plan prediction performance was tested on 16 new patients (2 pts per centre) extracting dose-volume-histogram (DVH) prediction bands of heart, ipsilateral lung, contralateral lung and breast. The inter-institutional variability was quantified by the standard deviations (SDint) of predicted DVHs and mean-dose (Dmean). The transferability of models, for the heart and the ipsilateral lung, was evaluated by the range of geometric Principal Component (PC1) applicability of a model to test patients of the other 7 institutions.

RESULTS

SDint of the DVH was 1.8 % and 1.6 % for the ipsilateral lung and the heart, respectively (20 %-80 % dose range); concerning Dmean, SDint was 0.9 Gy and 0.6 Gy for the ipsilateral lung and the heart, respectively (<0.2 Gy for contralateral organs). Mean predicted doses ranged between 4.3 and 5.9 Gy for the ipsilateral lung and 1.1-2.3 Gy for the heart. PC1 analysis suggested no relevant differences among models, except for one centre showing a systematic larger sparing of the heart, concomitant to a worse PTV coverage, due to high priority in sparing the left anterior descending coronary artery.

CONCLUSIONS

Results showed high transferability among models and low inter-institutional variability of 2% for plan prediction. These findings encourage the building of benchmark models in the case of TF-WBI.

Comparing Performances of Predictive Models of Toxicity after Radiotherapy for Breast Cancer Using Different Machine Learning Approaches

PURPOSE

Different ML models were compared to predict toxicity in RT on a large cohort (n = 1314).

METHODS

The endpoint was RTOG G2/G3 acute toxicity, resulting in 204/1314 patients with the event. The dataset, including 25 clinical, anatomical, and dosimetric features, was split into 984 for training and 330 for internal tests. The dataset was standardized; features with a high p-value at univariate LR and with Spearman ρ>0.8 were excluded; synthesized data of the minority were generated to compensate for class imbalance. Twelve ML methods were considered. Model optimization and sequential backward selection were run to choose the best models with a parsimonious feature number. Finally, feature importance was derived for every model.

RESULTS

The model's performance was compared on a training-test dataset over different metrics: the best performance model was LightGBM. Logistic regression with three variables (LR3) selected via bootstrapping showed performances similar to the best-performing models. The AUC of test data is slightly above 0.65 for the best models (highest value: 0.662 with LightGBM).

CONCLUSIONS

No model performed the best for all metrics: more complex ML models had better performances; however, models with just three features showed performances comparable to the best models using many (n = 13-19) features.

Reduction of SPECT acquisition time using deep learning: A phantom study

Single photon emission computed tomography (SPECT) procedures are characterized by long acquisition time to acquire diagnostically acceptable image data. The goal of this investigation was to assess the feasibility of using a deep convolutional neural network (DCNN) to reduce the acquisition time. The DCNN was implemented using the PyTorch and trained using image data from standard SPECT quality phantoms. The under-sampled image dataset is provided to neural network as input, while missing projections were provided as targets. The network is to produce for the output the missing projections. The baseline method of calculating the missing projections as arithmetic means of adjacent ones was introduced. The obtained synthesized projections and reconstructed images were compared to original data and baseline data across several parameters using PyTorch and PyTorch Image Quality code libraries. Results obtained from comparisons of projection and reconstructed image data show the DCNN clearly outperforming the baseline method. However, subsequent analysis revealed the synthesized image data being more comparable to under-sampled than to fully-sampled image data. The results of this investigation imply that neural network can replicate coarser objects better. However, densely sampled clinical image datasets, coarse reconstruction matrices and patient data featuring coarse structures combined with a lack of baseline data generation methods will hamper the ability to analyse the neural network outputs correctly. This study calls for use of phantom image data and introduction of a baseline method in the evaluation of neural network outputs.

External validation of an 18F-FDG-PET radiomic model predicting survival after radiotherapy for oropharyngeal cancer

PURPOSE/OBJECTIVE

The purpose of the study is to externally validate published ¹⁸F-FDG-PET radiomic models for outcome prediction in patients with oropharyngeal cancer treated with chemoradiotherapy.

MATERIAL/METHODS

Outcome data and pre-radiotherapy PET images of 100 oropharyngeal cancer patients (stage IV:78) treated with concomitant chemotherapy to 66-69 Gy/30 fr were available. Tumors were segmented using a previously validated semi-automatic method; 450 radiomic features (RF) were extracted according to IBSI (Image Biomarker Standardization Initiative) guidelines. Only one model for cancer-specific survival (CSS) prediction was suitable to be independently tested, according to our criteria. This model, in addition to HPV status, SUVmean and SUVmax, included two independent meta-factors (F_i), resulting from combining selected RF clusters. In a subgroup of 66 patients with complete HPV information, the global risk score R was computed considering the original coefficients and was tested by Cox regression as predictive of CSS. Independently, only the radiomic risk score R_F derived from F_i was tested on the same subgroup to learn about the radiomics contribution to the model. The metabolic tumor volume (MTV) was also tested as a single predictor and its prediction performances were compared to the global and radiomic models. Finally, the validation of MTV and the radiomic score R_F were also tested on the entire dataset.

RESULTS

Regarding the analysis of the subgroup with HPV information, with a median follow-up of 41.6 months, seven patients died due to cancer. R was confirmed to be associated to CSS (p value = 0.05) with a C-index equal 0.75 (95% CI=0.62-0.85). The best cut-off value (equal to 0.15) showed high ability in patient stratification (p=0.01, HR=7.4, 95% CI=1.6-11.4). The 5-year CSS for R were 97% (95% CI: 93-100%) vs 74% (56-92%) for low- and high-risk groups, respectively. R_F and MTV alone were also significantly associated to CSS for the subgroup with an almost identical C-index. According to best cut-off value (R_F>0.12 and MTV>15.5cc), the 5-year CSS were 96% (95% CI: 89-100%) vs 65% (36-94%) and 97% (95% CI: 88-100%) vs 77% (58-93%) for RF and MTV, respectively. Results regarding RF and MTV were confirmed in the overall group.

CONCLUSION

A previously published PET radiomic model for CSS prediction was independently validated. Performances of the model were similar to the ones of using only the MTV, without improvement of prediction accuracy.