Tissue-specific and interpretable sub-segmentation of whole tumour burden on CT images by unsupervised fuzzy clustering

Integrated radiogenomics models predict response to neoadjuvant chemotherapy in high grade serous ovarian cancer

High grade serous ovarian carcinoma (HGSOC) is a highly heterogeneous disease that typically presents at an advanced, metastatic state. The multi-scale complexity of HGSOC is a major obstacle to predicting response to neoadjuvant chemotherapy (NACT) and understanding critical determinants of response. Here we present a framework to predict the response of HGSOC patients to NACT integrating baseline clinical, blood-based, and radiomic biomarkers extracted from all primary and metastatic lesions. We use an ensemble machine learning model trained to predict the change in total disease volume using data obtained at diagnosis (n = 72). The model is validated in an internal hold-out cohort (n = 20) and an independent external patient cohort (n = 42). In the external cohort the integrated radiomics model reduces the prediction error by 8% with respect to the clinical model, achieving an AUC of 0.78 for RECIST 1.1 classification compared to 0.47 for the clinical model. Our results emphasize the value of including radiomics data in integrative models of treatment response and provide methods for developing new biomarker-based clinical trials of NACT in HGSOC.

CT-based deep learning segmentation of ovarian cancer and the stability of the extracted radiomics features

Background

Radiomics analysis could provide complementary tissue characterization in ovarian cancer (OC). However, OC segmentation required in radiomics analysis is time-consuming and labour-intensive. In this study, we aim to evaluate the performance of deep learning-based segmentation of OC on contrast-enhanced CT images and the stability of radiomics features extracted from the automated segmentation.

Methods

Staging abdominopelvic CT images of 367 patients with OC were retrospectively recruited. The training and cross-validation sets came from center A (n=283), and testing set (n=84) came from centers B and C. The tumours were manually delineated by a board-certified radiologist. Four model architectures provided by no-new-Net (nnU-Net) method were tested in this task. The segmentation performance evaluated by Dice score, Jaccard score, sensitivity and precision were compared among 4 architectures. The Pearson correlation coefficient (ρ), concordance correlation coefficient (ρc) and Bland-Altman plots were used to evaluate the volumetric assessment of OC between manual and automated segmentations. The stability of extracted radiomics features was evaluated by intraclass correlation coefficient (ICC).

Results

The 3D U-Net cascade architecture achieved highest median Dice score, Jaccard score, sensitivity and precision for OC segmentation in the testing set, 0.941, 0.890, 0.973 and 0.925, respectively. Tumour volumes of manual and automated segmentations were highly correlated (ρ=0.944 and ρc =0.933). 85.0% of radiomics features had high correlation with ICC >0.8.

Conclusions

The presented deep-learning segmentation could provide highly accurate automated segmentation of OC on CT images with high stability of the extracted radiomics features, showing the potential as a batch-processing segmentation tool.

Predictive Factors of Local Recurrence after Colorectal Cancer Liver Metastases Thermal Ablation

Background: Identify risk factors for local recurrence (LR) after radiofrequency (RFA) and microwave (MWA) thermoablations (TA) of colorectal cancer liver metastases (CCLM). Methods: Uni- (Pearson’s Chi2 test, Fisher’s exact test, Wilcoxon test) and multivariate analyses (LASSO logistic regressions) of every patient treated with MWA or RFA (percutaneously and surgically) from January 2015 to April 2021 in Centre Georges François Leclerc in Dijon, France. Results: Fifty-four patients were treated with TA for 177 CCLM (159 surgically, 18 percutaneously). LR rate was 17.5% of treated lesions. Univariate analyses by lesion showed factors associated with LR: sizes of the lesion (OR = 1.14), size of nearby vessel (OR = 1.27), treatment of a previous TA site LR (OR = 5.03), and non-ovoid TA site shape (OR = 4.25). Multivariate analyses showed that the size of the nearby vessel (OR = 1.17) and the lesion (OR = 1.09) remained significant risk factors of LR. Conclusions: The size of lesions to treat and vessel proximity are LR risk factors that need to be considered when making the decision of thermoablative treatments. TA of an LR on a previous TA site should be reserved to specific situations, as there is an important risk of another LR. An additional TA procedure can be discussed when TA site shape is non-ovoid on control imaging, given the risk of LR.

Radiomics and Radiogenomics of Ovarian Cancer

Ovarian cancer, one of the deadliest gynecologic malignancies, is characterized by high intra- and inter-site genomic and phenotypic heterogeneity. The traditional information provided by the conventional interpretation of diagnostic imaging studies cannot adequately represent this heterogeneity. Radiomics analyses can capture the complex patterns related to the microstructure of the tissues and provide quantitative information about them. This review outlines how radiomics and its integration with other quantitative biological information, like genomics and proteomics, can impact the clinical management of ovarian cancer.

Fully Automated Segmentation Models of Supratentorial Meningiomas Assisted by Inclusion of Normal Brain Images

To train an automatic brain tumor segmentation model, a large amount of data is required. In this paper, we proposed a strategy to overcome the limited amount of clinically collected magnetic resonance image (MRI) data regarding meningiomas by pre-training a model using a larger public dataset of MRIs of gliomas and augmenting our meningioma training set with normal brain MRIs. Pre-operative MRIs of 91 meningioma patients (171 MRIs) and 10 non-meningioma patients (normal brains) were collected between 2016 and 2019. Three-dimensional (3D) U-Net was used as the base architecture. The model was pre-trained with BraTS 2019 data, then fine-tuned with our datasets consisting of 154 meningioma MRIs and 10 normal brain MRIs. To increase the utility of the normal brain MRIs, a novel balanced Dice loss (BDL) function was used instead of the conventional soft Dice loss function. The model performance was evaluated using the Dice scores across the remaining 17 meningioma MRIs. The segmentation performance of the model was sequentially improved via the pre-training and inclusion of normal brain images. The Dice scores improved from 0.72 to 0.76 when the model was pre-trained. The inclusion of normal brain MRIs to fine-tune the model improved the Dice score; it increased to 0.79. When employing BDL as the loss function, the Dice score reached 0.84. The proposed learning strategy for U-net showed potential for use in segmenting meningioma lesions.

Artificial intelligence and machine learning in cancer imaging

An increasing array of tools is being developed using artificial intelligence (AI) and machine learning (ML) for cancer imaging. The development of an optimal tool requires multidisciplinary engagement to ensure that the appropriate use case is met, as well as to undertake robust development and testing prior to its adoption into healthcare systems. This multidisciplinary review highlights key developments in the field. We discuss the challenges and opportunities of AI and ML in cancer imaging; considerations for the development of algorithms into tools that can be widely used and disseminated; and the development of the ecosystem needed to promote growth of AI and ML in cancer imaging.

Neural-Symbolic Ensemble Learning for early-stage prediction of critical state of Covid-19 patients

Recently, Artificial Intelligence (AI) and Machine Learning (ML) have been successfully applied to many domains of interest including medical diagnosis. Due to the availability of a large quantity of data, it is possible to build reliable AI systems that assist humans in making decisions. The recent Covid-19 pandemic quickly spread over the world causing serious health problems and severe economic and social damage. Computer scientists are actively working together with doctors on different ML models to diagnose Covid-19 patients using Computed Tomography (CT) scans and clinical data. In this work, we propose a neural-symbolic system that predicts if a Covid-19 patient arriving at the hospital will end in a critical condition. The proposed system relies on Deep 3D Convolutional Neural Networks (3D-CNNs) for analyzing lung CT scans of Covid-19 patients, Decision Trees (DTs) for predicting if a Covid-19 patient will eventually pass away by analyzing its clinical data, and a neural system that integrates the previous ones using Hierarchical Probabilistic Logic Programs (HPLPs). Predicting if a Covid-19 patient will end in a critical condition is useful for managing the limited number of intensive care at the hospital. Moreover, knowing early that a Covid-19 patient could end in serious conditions allows doctors to gain early knowledge on patients and provide special treatment to those predicted to finish in critical conditions. The proposed system, entitled Neural HPLP, obtains good performance in terms of area under the receiver operating characteristic and precision curves with values of about 0.96 for both metrics. Therefore, with Neural HPLP, it is possible not only to efficiently predict if Covid-19 patients will end in severe conditions but also possible to provide an explanation of the prediction. This makes Neural HPLP explainable, interpretable, and reliable.

Graphical abstract

Early Prediction of Cerebral Computed Tomography under Intelligent Segmentation Algorithm Combined with Serological Indexes for Hematoma Enlargement after Intracerebral Hemorrhage

The aim of this study was to explore the application value of brain computed tomography (CT) images under intelligent segmentation algorithm and serological indexes in the early prediction of hematoma enlargement in patients with intracerebral hemorrhage (ICH). Fuzzy C-means (FCM) intelligence segmentation algorithm was introduced, and 150 patients with early ICH were selected as the research objects. Patient cerebral CT images were intelligently segmented to assess the diagnostic value of this algorithm. According to different hematoma volumes during CT examination, patients were divided into observation group (hematoma enlargement occurred, n = 48) and control group (no hematoma enlargement occurred, n = 102). The predicative value of hematoma enlargement after ICH was investigated by assessing CT image quality and measuring intracerebral edema, hematoma volume, and serological indicators of the patients of the two groups. The results demonstrated that the sensitivity, specificity, and accuracy of CT images processed by intelligence segmentation algorithm amounted to 0.894, 0.898, and 0.930, respectively. Besides, early edema enlargement and hematoma of patients in the observation group were more significant than those of patients in the control group. Relative edema volume was 0.912, which was apparently lower than that in the control group (1.017) (P < 0.05). In terms of CT signs of ICH patients, the incidence of blend sign, low density sign, and stroke of the observation group was evidently higher than those of the control group (P < 0.05). Besides, absolute lymphocyte count (ALC) and hemoglobin (HGB) concentration of the patients in the observation group were 6.23 × 109/L and 6.29 × 109/L, respectively, both of which were higher than those of the control group (6.08 × 109/L and 4.25 × 109/L). Neutrophil to lymphocyte ratio (NLR) was 0.99 × 109/L, which was apparently lower than that in the control group (1.43 × 109/L) (P < 0.05). To sum up, cerebral CT images processed by FCM algorithm showed good diagnostic effect on ICH and high clinical values in the early prediction of hematoma among ICH patients.

Clinically Interpretable Radiomics-Based Prediction of Histopathologic Response to Neoadjuvant Chemotherapy in High-Grade Serous Ovarian Carcinoma

Background

Pathological response to neoadjuvant treatment for patients with high-grade serous ovarian carcinoma (HGSOC) is assessed using the chemotherapy response score (CRS) for omental tumor deposits. The main limitation of CRS is that it requires surgical sampling after initial neoadjuvant chemotherapy (NACT) treatment. Earlier and non-invasive response predictors could improve patient stratification. We developed computed tomography (CT) radiomic measures to predict neoadjuvant response before NACT using CRS as a gold standard.

Methods

Omental CT-based radiomics models, yielding a simplified fully interpretable radiomic signature, were developed using Elastic Net logistic regression and compared to predictions based on omental tumor volume alone. Models were developed on a single institution cohort of neoadjuvant-treated HGSOC (n = 61; 41% complete response to NCT) and tested on an external test cohort (n = 48; 21% complete response).

Results

The performance of the comprehensive radiomics models and the fully interpretable radiomics model was significantly higher than volume-based predictions of response in both the discovery and external test sets when assessed using G-mean (geometric mean of sensitivity and specificity) and NPV, indicating high generalizability and reliability in identifying non-responders when using radiomics. The performance of a fully interpretable model was similar to that of comprehensive radiomics models.

Conclusions

CT-based radiomics allows for predicting response to NACT in a timely manner and without the need for abdominal surgery. Adding pre-NACT radiomics to volumetry improved model performance for predictions of response to NACT in HGSOC and was robust to external testing. A radiomic signature based on five robust predictive features provides improved clinical interpretability and may thus facilitate clinical acceptance and application.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).

Automatic Liver Segmentation in CT Images with Enhanced GAN and Mask Region-Based CNN Architectures

Liver image segmentation has been increasingly employed for key medical purposes, including liver functional assessment, disease diagnosis, and treatment. In this work, we introduce a liver image segmentation method based on generative adversarial networks (GANs) and mask region-based convolutional neural networks (Mask R-CNN). Firstly, since most resulting images have noisy features, we further explored the combination of Mask R-CNN and GANs in order to enhance the pixel-wise classification. Secondly, k-means clustering was used to lock the image aspect ratio, in order to get more essential anchors which can help boost the segmentation performance. Finally, we proposed a GAN Mask R-CNN algorithm which achieved superior performance in comparison with the conventional Mask R-CNN, Mask-CNN, and k-means algorithms in terms of the Dice similarity coefficient (DSC) and the MICCAI metrics. The proposed algorithm also achieved superior performance in comparison with ten state-of-the-art algorithms in terms of six Boolean indicators. We hope that our work can be effectively used to optimize the segmentation and classification of liver anomalies.

Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation

Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentation are based on either the cross entropy loss, Dice loss or a combination of the two. We propose the Unified Focal loss, a new hierarchical framework that generalises Dice and cross entropy-based losses for handling class imbalance. We evaluate our proposed loss function on five publicly available, class imbalanced medical imaging datasets: CVC-ClinicDB, Digital Retinal Images for Vessel Extraction (DRIVE), Breast Ultrasound 2017 (BUS2017), Brain Tumour Segmentation 2020 (BraTS20) and Kidney Tumour Segmentation 2019 (KiTS19). We compare our loss function performance against six Dice or cross entropy-based loss functions, across 2D binary, 3D binary and 3D multiclass segmentation tasks, demonstrating that our proposed loss function is robust to class imbalance and consistently outperforms the other loss functions. Source code is available at: https://github.com/mlyg/unified-focal-loss.

•

Loss function choice is crucial for class-imbalanced medical imaging datasets.

•

Understanding the relationship between loss functions is key to inform choice.

•

Unified Focal loss generalises Dice and cross-entropy based loss functions.

•

Unified Focal loss outperforms various Dice and cross-entropy based loss functions.

An automated liver segmentation in liver iron concentration map using fuzzy c-means clustering combined with anatomical landmark data

Background

To estimate median liver iron concentration (LIC) calculated from magnetic resonance imaging, excluded vessels of the liver parenchyma region were defined manually. Previous works proposed the automated method for excluding vessels from the liver region. However, only user-defined liver region remained a manual process. Therefore, this work aimed to develop an automated liver region segmentation technique to automate the whole process of median LIC calculation.

Methods

553 MR examinations from 471 thalassemia major patients were used in this study. LIC maps (in mg/g dry weight) were calculated and used as the input of segmentation procedures. Anatomical landmark data were detected and used to restrict ROI. After that, the liver region was segmented using fuzzy c-means clustering and reduced segmentation errors by morphological processes. According to the clinical application, erosion with a suitable size of the structuring element was applied to reduce the segmented liver region to avoid uncertainty around the edge of the liver. The segmentation results were evaluated by comparing with manual segmentation performed by a board-certified radiologist.

Results

The proposed method was able to produce a good grade output in approximately 81% of all data. Approximately 11% of all data required an easy modification step. The rest of the output, approximately 8%, was an unsuccessful grade and required manual intervention by a user. For the evaluation matrices, percent dice similarity coefficient (%DSC) was in the range 86–92, percent Jaccard index (%JC) was 78–86, and Hausdorff distance (H) was 14–28 mm, respectively. In this study, percent false positive (%FP) and percent false negative (%FN) were applied to evaluate under- and over-segmentation that other evaluation matrices could not handle. The average of operation times could be reduced from 10 s per case using traditional method, to 1.5 s per case using our proposed method.

Conclusion

The experimental results showed that the proposed method provided an effective automated liver segmentation technique, which can be applied clinically for automated median LIC calculation in thalassemia major patients.

Cascaded Regression Neural Nets for Kidney Localization and Segmentation-free Volume Estimation

Kidney volume is an essential biomarker for a number of kidney disease diagnoses, for example, chronic kidney disease. Existing total kidney volume estimation methods often rely on an intermediate kidney segmentation step. On the other hand, automatic kidney localization in volumetric medical images is a critical step that often precedes subsequent data processing and analysis. Most current approaches perform kidney localization via an intermediate classification or regression step. This paper proposes an integrated deep learning approach for (i) kidney localization in computed tomography scans and (ii) segmentation-free renal volume estimation. Our localization method uses a selection-convolutional neural network that approximates the kidney inferior-superior span along the axial direction. Cross-sectional (2D) slices from the estimated span are subsequently used in a combined sagittal-axial Mask-RCNN that detects the organ bounding boxes on the axial and sagittal slices, the combination of which produces a final 3D organ bounding box. Furthermore, we use a fully convolutional network to estimate the kidney volume that skips the segmentation procedure. We also present a mathematical expression to approximate the 'volume error' metric from the 'Sørensen-Dice coefficient.' We accessed 100 patients' CT scans from the Vancouver General Hospital records and obtained 210 patients' CT scans from the 2019 Kidney Tumor Segmentation Challenge database to validate our method. Our method produces a kidney boundary wall localization error of ~2.4mm and a mean volume estimation error of ~5%.

Ultrasound-guided targeted biopsies of CT-based radiomic tumour habitats: technical development and initial experience in metastatic ovarian cancer

PURPOSE

To develop a precision tissue sampling technique that uses computed tomography (CT)-based radiomic tumour habitats for ultrasound (US)-guided targeted biopsies that can be integrated in the clinical workflow of patients with high-grade serous ovarian cancer (HGSOC).

METHODS

Six patients with suspected HGSOC scheduled for US-guided biopsy before starting neoadjuvant chemotherapy were included in this prospective study from September 2019 to February 2020. The tumour segmentation was performed manually on the pre-biopsy contrast-enhanced CT scan. Spatial radiomic maps were used to identify tumour areas with similar or distinct radiomic patterns, and tumour habitats were identified using the Gaussian mixture modelling. CT images with superimposed habitat maps were co-registered with US images by means of a landmark-based rigid registration method for US-guided targeted biopsies. The dice similarity coefficient (DSC) was used to assess the tumour-specific CT/US fusion accuracy.

RESULTS

We successfully co-registered CT-based radiomic tumour habitats with US images in all patients. The median time between CT scan and biopsy was 21 days (range 7-30 days). The median DSC for tumour-specific CT/US fusion accuracy was 0.53 (range 0.79 to 0.37). The CT/US fusion accuracy was high for the larger pelvic tumours (DSC: 0.76-0.79) while it was lower for the smaller omental metastases (DSC: 0.37-0.53).

CONCLUSION

We developed a precision tissue sampling technique that uses radiomic habitats to guide in vivo biopsies using CT/US fusion and that can be seamlessly integrated in the clinical routine for patients with HGSOC.

KEY POINTS

• We developed a prevision tissue sampling technique that co-registers CT-based radiomics-based tumour habitats with US images. • The CT/US fusion accuracy was high for the larger pelvic tumours (DSC: 0.76-0.79) while it was lower for the smaller omental metastases (DSC: 0.37-0.53).

Boundary Loss-Based 2.5D Fully Convolutional Neural Networks Approach for Segmentation: A Case Study of the Liver and Tumor on Computed Tomography

Image segmentation plays an important role in the field of image processing, helping to understand images and recognize objects. However, most existing methods are often unable to effectively explore the spatial information in 3D image segmentation, and they neglect the information from the contours and boundaries of the observed objects. In addition, shape boundaries can help to locate the positions of the observed objects, but most of the existing loss functions neglect the information from the boundaries. To overcome these shortcomings, this paper presents a new cascaded 2.5D fully convolutional networks (FCNs) learning framework to segment 3D medical images. A new boundary loss that incorporates distance, area, and boundary information is also proposed for the cascaded FCNs to learning more boundary and contour features from the 3D medical images. Moreover, an effective post-processing method is developed to further improve the segmentation accuracy. We verified the proposed method on LITS and 3DIRCADb datasets that include the liver and tumors. The experimental results show that the performance of the proposed method is better than existing methods with a Dice Per Case score of 74.5% for tumor segmentation, indicating the effectiveness of the proposed method.

Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991-2020)

Understanding the data and reaching accurate conclusions are of paramount importance in the present era of big data. Machine learning and probability theory methods have been widely used for this purpose in various fields. One critically important yet less explored aspect is capturing and analyzing uncertainties in the data and model. Proper quantification of uncertainty helps to provide valuable information to obtain accurate diagnosis. This paper reviewed related studies conducted in the last 30 years (from 1991 to 2020) in handling uncertainties in medical data using probability theory and machine learning techniques. Medical data is more prone to uncertainty due to the presence of noise in the data. So, it is very important to have clean medical data without any noise to get accurate diagnosis. The sources of noise in the medical data need to be known to address this issue. Based on the medical data obtained by the physician, diagnosis of disease, and treatment plan are prescribed. Hence, the uncertainty is growing in healthcare and there is limited knowledge to address these problems. Our findings indicate that there are few challenges to be addressed in handling the uncertainty in medical raw data and new models. In this work, we have summarized various methods employed to overcome this problem. Nowadays, various novel deep learning techniques have been proposed to deal with such uncertainties and improve the performance in decision making.

Kidney and Renal Tumor Segmentation Using a Hybrid V-Net-Based Model

Kidney tumors represent a type of cancer that people of advanced age are more likely to develop. For this reason, it is important to exercise caution and provide diagnostic tests in the later stages of life. Medical imaging and deep learning methods are becoming increasingly attractive in this sense. Developing deep learning models to help physicians identify tumors with successful segmentation is of great importance. However, not many successful systems exist for soft tissue organs, such as the kidneys and the prostate, of which segmentation is relatively difficult. In such cases where segmentation is difficult, V-Net-based models are mostly used. This paper proposes a new hybrid model using the superior features of existing V-Net models. The model represents a more successful system with improvements in the encoder and decoder phases not previously applied. We believe that this new hybrid V-Net model could help the majority of physicians, particularly those focused on kidney and kidney tumor segmentation. The proposed model showed better performance in segmentation than existing imaging models and can be easily integrated into all systems due to its flexible structure and applicability. The hybrid V-Net model exhibited average Dice coefficients of 97.7% and 86.5% for kidney and tumor segmentation, respectively, and, therefore, could be used as a reliable method for soft tissue organ segmentation.