Automatic tumor segmentation using knowledge-based techniques

Signal-to-Noise Ratio Imaging and Real-Time Sharpening of Tumor Boundaries for Image-Guided Cancer Surgery

Fluorescence-guided cancer surgery is of considerable current interest in bioanalytical chemistry, engineering, and medicine, but its clinical utility is still hampered by the diffusive (scattering) nature of human tissues and large variations among different patients. Here, we report a new method based on signal-to-noise (contrast-to-noise) ratio (SNR or CNR) imaging for real-time delineation and sharpening of tumor boundaries during image-guided cancer surgery. In particular, we show that in vivo tumor fluorescence signals (both intensity and standard deviation) are strongly correlated with those of the surrounding tissue of the same tissue type and that this relationship is maintained as a function of time for fluorescent tracers such as indocyanine green. This dynamic relationship permits a precise removal of nonspecific background fluorescence from tumor fluorescence. As a result, single-pixel SNR values have been calculated, mapped, and displayed across a large surgical field at 60 frames per second. Pathological validation studies indicate that these SNR values correspond to statistical confidence levels similar (but not identical) to those of normal distributions. When the tumor fluorescence has an SNR of 3, pathological data show a confidence level of approximately 95% in identifying the true tumor lesions. For clinical relevance, we have also carried out first-in-human clinical studies for both oral and esophageal tumors, achieving tumor margin precisions of 1-2 mm with 87.5% histological accuracy and no false positives.

Automatic segmentation of brain glioma based on XY-Net

Glioma is a malignant primary brain tumor, which can easily lead to death if it is not detected in time. Magnetic resonance imaging is the most commonly used technique to diagnose gliomas, and precise outlining of tumor areas from magnetic resonance images (MRIs) is an important aid to physicians in understanding the patient's condition and formulating treatment plans. However, relying on radiologists to manually depict tumors is a tedious and laborious task, so it is clinically important to investigate an automated method for outlining glioma regions in MRIs. To liberate radiologists from the heavy task of outlining tumors, we propose a fully convolutional network, XY-Net, based on the most popular U-Net symmetric encoder-decoder structure to perform automatic segmentation of gliomas. We construct two symmetric sub-encoders for XY-Net and build interconnected X-shaped feature map transmission paths between the sub-encoders, while maintaining the feature map concatenation between each sub-encoder and the decoder. Moreover, a loss function composed of the balanced cross-entropy loss function and the dice loss function is used in the training task of XY-Net to solve the class unevenness problem of the medical image segmentation task. The experimental results show that the proposed XY-Net has a 2.16% improvement in dice coefficient (DC) compared to the network model with a single encoder structure, and compare with some state-of-the-art image segmentation methods, XY-Net achieves the best performance. The DC, HD, recall, and precision of our method on the test set are 74.49%, 10.89 mm, 78.06%, and 76.30%, respectively. The combination of sub-encoders and cross-transmission paths enables the model to perform better; based on this combination, the XY-Net achieves an end-to-end automatic segmentation of gliomas on 2D slices of MRIs, which can play a certain auxiliary role for doctors in grasping the state of illness.

A review on brain tumor segmentation based on deep learning methods with federated learning techniques

Brain tumors have become a severe medical complication in recent years due to their high fatality rate. Radiologists segment the tumor manually, which is time-consuming, error-prone, and expensive. In recent years, automated segmentation based on deep learning has demonstrated promising results in solving computer vision problems such as image classification and segmentation. Brain tumor segmentation has recently become a prevalent task in medical imaging to determine the tumor location, size, and shape using automated methods. Many researchers have worked on various machine and deep learning approaches to determine the most optimal solution using the convolutional methodology. In this review paper, we discuss the most effective segmentation techniques based on the datasets that are widely used and publicly available. We also proposed a survey of federated learning methodologies to enhance global segmentation performance and ensure privacy. A comprehensive literature review is suggested after studying more than 100 papers to generalize the most recent techniques in segmentation and multi-modality information. Finally, we concentrated on unsolved problems in brain tumor segmentation and a client-based federated model training strategy. Based on this review, future researchers will understand the optimal solution path to solve these issues.

Qutrit-Inspired Fully Self-Supervised Shallow Quantum Learning Network for Brain Tumor Segmentation

Classical self-supervised networks suffer from convergence problems and reduced segmentation accuracy due to forceful termination. Qubits or bilevel quantum bits often describe quantum neural network models. In this article, a novel self-supervised shallow learning network model exploiting the sophisticated three-level qutrit-inspired quantum information system, referred to as quantum fully self-supervised neural network (QFS-Net), is presented for automated segmentation of brain magnetic resonance (MR) images. The QFS-Net model comprises a trinity of a layered structure of qutrits interconnected through parametric Hadamard gates using an eight-connected second-order neighborhood-based topology. The nonlinear transformation of the qutrit states allows the underlying quantum neural network model to encode the quantum states, thereby enabling a faster self-organized counterpropagation of these states between the layers without supervision. The suggested QFS-Net model is tailored and extensively validated on the Cancer Imaging Archive (TCIA) dataset collected from the Nature repository. The experimental results are also compared with state-of-the-art supervised (U-Net and URes-Net architectures) and the self-supervised QIS-Net model and its classical counterpart. Results shed promising segmented outcomes in detecting tumors in terms of dice similarity and accuracy with minimum human intervention and computational resources. The proposed QFS-Net is also investigated on natural gray-scale images from the Berkeley segmentation dataset and yields promising outcomes in segmentation, thereby demonstrating the robustness of the QFS-Net model.

Deep learning based brain tumor segmentation: a survey

Brain tumor segmentation is one of the most challenging problems in medical image analysis. The goal of brain tumor segmentation is to generate accurate delineation of brain tumor regions. In recent years, deep learning methods have shown promising performance in solving various computer vision problems, such as image classification, object detection and semantic segmentation. A number of deep learning based methods have been applied to brain tumor segmentation and achieved promising results. Considering the remarkable breakthroughs made by state-of-the-art technologies, we provide this survey with a comprehensive study of recently developed deep learning based brain tumor segmentation techniques. More than 150 scientific papers are selected and discussed in this survey, extensively covering technical aspects such as network architecture design, segmentation under imbalanced conditions, and multi-modality processes. We also provide insightful discussions for future development directions.

Automated knowledge-assisted mitosis cells detection framework in breast histopathology images

Based on the Nottingham Histopathology Grading (NHG) system, mitosis cells detection is one of the important criteria to determine the grade of breast carcinoma. Mitosis cells detection is a challenging task due to the heterogeneous microenvironment of breast histopathology images. Recognition of complex and inconsistent objects in the medical images could be achieved by incorporating domain knowledge in the field of interest. In this study, the strategies of the histopathologist and domain knowledge approach were used to guide the development of the image processing framework for automated mitosis cells detection in breast histopathology images. The detection framework starts with color normalization and hyperchromatic nucleus segmentation. Then, a knowledge-assisted false positive reduction method is proposed to eliminate the false positive (i.e., non-mitosis cells). This stage aims to minimize the percentage of false positive and thus increase the F1-score. Next, features extraction was performed. The mitosis candidates were classified using a Support Vector Machine (SVM) classifier. For evaluation purposes, the knowledge-assisted detection framework was tested using two datasets: a custom dataset and a publicly available dataset (i.e., MITOS dataset). The proposed knowledge-assisted false positive reduction method was found promising by eliminating at least 87.1% of false positive in both the dataset producing promising results in the F1-score. Experimental results demonstrate that the knowledge-assisted detection framework can achieve promising results in F1-score (custom dataset: 89.1%; MITOS dataset: 88.9%) and outperforms the recent works.

An efficient brain tumor image classifier by combining multi-pathway cascaded deep neural network and handcrafted features in MR images

Accurate segmentation and delineation of the sub-tumor regions are very challenging tasks due to the nature of the tumor. Traditionally, convolutional neural networks (CNNs) have succeeded in achieving most promising performance for the segmentation of brain tumor; however, handcrafted features remain very important in identification of tumor's boundary regions accurately. The present work proposes a robust deep learning-based model with three different CNN architectures along with pre-defined handcrafted features for brain tumor segmentation, mainly to find out more prominent boundaries of the core and enhanced tumor regions. Generally, automatic CNN architecture does not use the pre-defined handcrafted features because it extracts the features automatically. In this present work, several pre-defined handcrafted features are computed from four MRI modalities (T2, FLAIR, T1c, and T1) with the help of additional handcrafted masks according to user interest and fed to the convolutional features (automatic features) to improve the overall performance of the proposed CNN model for tumor segmentation. Multi-pathway CNN is explored in this present work along with single-pathway CNN, which extracts simultaneously both local and global features to identify the accurate sub-regions of the tumor with the help of handcrafted features. The present work uses a cascaded CNN architecture, where the outcome of a CNN is considered as an additional input information to next subsequent CNNs. To extract the handcrafted features, convolutional operation was applied on the four MRI modalities with the help of several pre-defined masks to produce a predefined set of handcrafted features. The present work also investigates the usefulness of intensity normalization and data augmentation in pre-processing stage in order to handle the difficulties related to the imbalance of tumor labels. The proposed method is experimented on the BraST 2018 datasets and achieved promising results than the existing (currently published) methods with respect to different metrics such as specificity, sensitivity, and dice similarity coefficient (DSC) for complete, core, and enhanced tumor regions. Quantitatively, a notable gain is achieved around the boundaries of the sub-tumor regions using the proposed two-pathway CNN along with the handcrafted features. Graphical Abstract This data is mandatory. Please provide.

A novel method for segmenting brain tumor using modified watershed algorithm in MRI image with FPGA

The goal of the segmentation of brain images is to separate the images in different non-compatible homogenous areas reflecting the numerous anatomical structures. Brain segmentation by magnetic resonance has numerous implications for diagnosing brain disorganizations such as Alzheimer's, Parkinson-related syndrome among others. However, it is not an simple job to automatically segment the MR image. The main motive of this study is to provide a better segmentation approach for the segment of the ROI (Region Of Interest) region from the MRI image by solving the issues that currently exist in the literary works. MRI segmentation is not a trivial task, because acquired MR images are imperfect and are often corrupted by noise and other image artifacts. The variety in technologies for image processing has contributed to the creation in numerous image segmentation techniques. That is because there is no universal approach, nor are all methods necessarily appropriate for a specific form of picture suitable for all pictures. Other approaches still use the gray level histogram, for example, while others integrate detailed spatial picture details for bleeding conditions. Some methods use statistical techniques, but some do incorporate existing information to enhance segmentation efficiency. Some methods utilize probabilistic or fuzzy methods. Yet there are certain inconveniences of all the current processes. Therefore, we have intended to propose a new segmentation approach for the ROI region segmentation. The proposed work comprised of three phases namely preprocessing, edge detection and segmentation. At first, the MRI images are extracted from the database and that each of the input images is enhanced by applying a high pass filter. After completing the preprocessing method, the enhanced canny edge detection (ECED) approach is used to enhance the image. After that, the images are given to the modified watershed segmentation (MWS) algorithm which separates the ROI part from MRI Image. The testing consequences demonstrate that the proposed system accomplishes to give the good result related to the available strategies. Xilinx Virtex-5 FPGA is used to implement in this paper.

Multimodal MRI Brain Tumor Image Segmentation Using Sparse Subspace Clustering Algorithm

Brain tumors are one of the most deadly diseases with a high mortality rate. The shape and size of the tumor are random during the growth process. Brain tumor segmentation is a brain tumor assisted diagnosis technology that separates different brain tumor structures such as edema and active and tumor necrosis tissues from normal brain tissue. Magnetic resonance imaging (MRI) technology has the advantages of no radiation impact on the human body, good imaging effect on structural tissues, and an ability to realize tomographic imaging of any orientation. Therefore, doctors often use MRI brain tumor images to analyze and process brain tumors. In these images, the tumor structure is only characterized by grayscale changes, and the developed images obtained by different equipment and different conditions may also be different. This makes it difficult for traditional image segmentation methods to deal well with the segmentation of brain tumor images. Considering that the traditional single-mode MRI brain tumor images contain incomplete brain tumor information, it is difficult to segment the single-mode brain tumor images to meet clinical needs. In this paper, a sparse subspace clustering (SSC) algorithm is introduced to process the diagnosis of multimodal MRI brain tumor images. In the absence of added noise, the proposed algorithm has better advantages than traditional methods. Compared with the top 15 in the Brats 2015 competition, the accuracy is not much different, being basically stable between 10 and 15. In order to verify the noise resistance of the proposed algorithm, this paper adds 5%, 10%, 15%, and 20% Gaussian noise to the test image. Experimental results show that the proposed algorithm has better noise immunity than a comparable algorithm.

Fully automatic brain tumor segmentation with deep learning-based selective attention using overlapping patches and multi-class weighted cross-entropy

In this paper, we present a new Deep Convolutional Neural Networks (CNNs) dedicated to fully automatic segmentation of Glioblastoma brain tumors with high- and low-grade. The proposed CNNs model is inspired by the Occipito-Temporal pathway which has a special function called selective attention that uses different receptive field sizes in successive layers to figure out the crucial objects in a scene. Thus, using selective attention technique to develop the CNNs model, helps to maximize the extraction of relevant features from MRI images. We have also addressed two more issues: class-imbalance, and the spatial relationship among image Patches. To address the first issue, we propose two steps: an equal sampling of images Patches and an experimental analysis of the effect of weighted cross-entropy loss function on the segmentation results. In addition, to overcome the second issue, we have studied the effect of Overlapping Patches against Adjacent Patches where the Overlapping Patches show better segmentation results due to the introduction of the global context as well as the local features of the image Patches compared to the conventionnel Adjacent Patches. Our experiment results are reported on BRATS-2018 dataset where our End-to-End Deep Learning model achieved state-of-the-art performance. The median Dice score of our fully automatic segmentation model is 0.90, 0.83, 0.83 for the whole tumor, tumor core, and enhancing tumor respectively compared to the Dice score of radiologist, that is in the range 74% - 85%. Moreover, our proposed CNNs model is not only computationally efficient at inference time, but it could segment the whole brain on average 12 seconds. Finally, the proposed Deep Learning model provides an accurate and reliable segmentation result, and that makes it suitable for adopting in research and as a part of different clinical settings.

External validation and transfer learning of convolutional neural networks for computed tomography dental artifact classification

Quality assurance of data prior to use in automated pipelines and image analysis would assist in safeguarding against biases and incorrect interpretation of results. Automation of quality assurance steps would further improve robustness and efficiency of these methods, motivating widespread adoption of techniques. Previous work by our group demonstrated the ability of convolutional neural networks (CNN) to efficiently classify head and neck (H&N) computed-tomography (CT) images for the presence of dental artifacts (DA) that obscure visualization of structures and the accuracy of Hounsfield units. In this work we demonstrate the generalizability of our previous methodology by validating CNNs on six external datasets, and the potential benefits of transfer learning with fine-tuning on CNN performance. 2112 H&N CT images from seven institutions were scored as DA positive or negative. 1538 images from a single institution were used to train three CNNs with resampling grid sizes of 64³, 128³ and 256³. The remaining six external datasets were used in five-fold cross-validation with a data split of 20% training/fine-tuning and 80% validation. The three pre-trained models were each validated using the five-folds of the six external datasets. The pre-trained models also underwent transfer learning with fine-tuning using the 20% training/fine-tuning data, and validated using the corresponding validation datasets. The highest micro-averaged AUC for our pre-trained models across all external datasets occurred with a resampling grid of 256³ (AUC  =  0.91  ±  0.01). Transfer learning with fine-tuning improved generalizability when utilizing a resampling grid of 256³ to a micro-averaged AUC of 0.92  ±  0.01. Despite these promising results, transfer learning did not improve AUC when utilizing small resampling grids or small datasets. Our work demonstrates the potential of our previously developed automated quality assurance methods to generalize to external datasets. Additionally, we showed that transfer learning with fine-tuning using small portions of external datasets can be used to fine-tune models for improved performance when large variations in images are present.

Brain Tumor Detection by Using Stacked Autoencoders in Deep Learning

Brain tumor detection depicts a tough job because of its shape, size and appearance variations. In this manuscript, a deep learning model is deployed to predict input slices as a tumor (unhealthy)/non-tumor (healthy). This manuscript employs a high pass filter image to prominent the inhomogeneities field effect of the MR slices and fused with the input slices. Moreover, the median filter is applied to the fused slices. The resultant slices quality is improved with smoothen and highlighted edges of the input slices. After that, based on these slices' intensity, a 4-connected seed growing algorithm is applied, where optimal threshold clusters the similar pixels from the input slices. The segmented slices are then supplied to the fine-tuned two layers proposed stacked sparse autoencoder (SSAE) model. The hyperparameters of the model are selected after extensive experiments. At the first layer, 200 hidden units and at the second layer 400 hidden units are utilized. The testing is performed on the softmax layer for the prediction of the images having tumors and no tumors. The suggested model is trained and checked on BRATS datasets i.e., 2012(challenge and synthetic), 2013, and 2013 Leaderboard, 2014, and 2015 datasets. The presented model is evaluated with a number of performance metrics which demonstrates the improved performance.

Optimization of Brain Tumor MR Image Classification Accuracy Using Optimal Threshold, PCA and Training ANFIS with Different Repetitions

BACKGROUND

One of the leading causes of death is brain tumors. Accurate tumor classification leads to appropriate decision making and providing the most efficient treatment to the patients. This study aims to optimize brain tumor MR images classification accuracy using optimal threshold, PCA and training Adaptive Neuro Fuzzy Inference System (ANFIS) with different repetitions.

MATERIAL AND METHODS

The procedure used in this study consists of five steps: (1) T1, T2 weighted images collection, (2) tumor separation with different threshold levels, (3) feature extraction, (4) presence and absence of feature reduction applying principal component analysis (PCA) and (5) ANFIS classification with 0, 20 and 200 training repetitions.

RESULTS

ANFIS accuracy was 40%, 80% and 97% for all features and 97%, 98.5% and 100% for the 6 selected features by PCA in 0, 20 and 200 training repetitions, respectively.

CONCLUSION

The findings of the present study demonstrated that accuracy can be raised up to 100% by using an optimized threshold method, PCA and increasing training repetitions.

The Applications of Radiomics in Precision Diagnosis and Treatment of Oncology: Opportunities and Challenges

Medical imaging can assess the tumor and its environment in their entirety, which makes it suitable for monitoring the temporal and spatial characteristics of the tumor. Progress in computational methods, especially in artificial intelligence for medical image process and analysis, has converted these images into quantitative and minable data associated with clinical events in oncology management. This concept was first described as radiomics in 2012. Since then, computer scientists, radiologists, and oncologists have gravitated towards this new tool and exploited advanced methodologies to mine the information behind medical images. On the basis of a great quantity of radiographic images and novel computational technologies, researchers developed and validated radiomic models that may improve the accuracy of diagnoses and therapy response assessments. Here, we review the recent methodological developments in radiomics, including data acquisition, tumor segmentation, feature extraction, and modelling, as well as the rapidly developing deep learning technology. Moreover, we outline the main applications of radiomics in diagnosis, treatment planning and evaluations in the field of oncology with the aim of developing quantitative and personalized medicine. Finally, we discuss the challenges in the field of radiomics and the scope and clinical applicability of these methods.

MRI Brain Tumour Segmentation Using Hybrid Clustering and Classification by Back Propagation Algorithm

Generally the segmentation refers, the partitioning of an image into smaller regions to identify or locate the region of abnormality. Even though image segmentation is the challenging task in medical applications, due to contrary image, local observations of an image, noise image, non uniform texture of the images and so on. Many techniques are available for image segmentation, but still it requires to introduce an efficient, fast medical image segmentation methods. This research article introduces an efficient image segmentation method based on K means clustering integrated with a spatial Fuzzy C means clustering algorithms. The suggested technique combines the advantages of the two methods. K means segmentation requires minimum computation time, but spatial Fuzzy C means provides high accuracy for image segmentation. The performance of the proposed method is evaluated in terms of accuracy, PSNR and processing time. It also provides good implementation results for MRI brain image segmentation with high accuracy and minimal execution time. After completing the segmentation the of abnormal part of the input MRI brain image, it is compulsory to classify the image is normal or abnormal. There are many classifiers like a self organizing map, Back propagation algorithm, support vector machine etc., The algorithm helps to classify the abnormalities like benign or malignant brain tumour in case of MRI brain image. The abnormality is detected based on the extracted features from an input image. Discrete wavelet transform helps to find the hidden information from the MRI brain image. The extracted features are trained by Back Propagation Algorithm to classify the abnormalities of MRI brain image.

Fully Automatic Brain Tumor Segmentation using End-To-End Incremental Deep Neural Networks in MRI images

BACKGROUND AND OBJECTIVE

Nowadays, getting an efficient Brain Tumor Segmentation in Multi-Sequence MR images as soon as possible, gives an early clinical diagnosis, treatment and follow-up. The aim of this study is to develop a new deep learning model for the segmentation of brain tumors. The proposed models are used to segment the brain tumors of Glioblastomas (with both high and low grade). Glioblastomas have four properties: different sizes, shapes, contrasts, in addition, Glioblastomas appear anywhere in the brain.

METHODS

In this paper, we propose three end-to-end Incremental Deep Convolutional Neural Networks models for fully automatic Brain Tumor Segmentation. Our proposed models are different from the other CNNs-based models that follow the technique of trial and error process which does not use any guided approach to get the suitable hyper-parameters. Moreover, we adopt the technique of Ensemble Learning to design a more efficient model. For solving the problem of training CNNs model, we propose a new training strategy which takes into account the most influencing hyper-parameters by bounding and setting a roof to these hyper-parameters to accelerate the training.

RESULTS

Our experiment results reported on BRATS-2017 dataset. The proposed deep learning models achieve the state-of-the-art performance without any post-processing operations. Indeed, our models achieve in average 0.88 Dice score over the complete region. Moreover, the efficient design with the advantage of GPU implementation, allows our three deep learning models to achieve brain segmentation results in average 20.87 s.

CONCLUSIONS

The proposed deep learning models are effective for the segmentation of brain tumors and allow to obtain high accurate results. Moreover, the proposed models could help the physician experts to reduce the time of diagnostic.

Artificial intelligence in radiation oncology: A specialty-wide disruptive transformation?

Artificial intelligence (AI) is emerging as a technology with the power to transform established industries, and with applications from automated manufacturing to advertising and facial recognition to fully autonomous transportation. Advances in each of these domains have led some to call AI the "fourth" industrial revolution [1]. In healthcare, AI is emerging as both a productive and disruptive force across many disciplines. This is perhaps most evident in Diagnostic Radiology and Pathology, specialties largely built around the processing and complex interpretation of medical images, where the role of AI is increasingly seen as both a boon and a threat. In Radiation Oncology as well, AI seems poised to reshape the specialty in significant ways, though the impact of AI has been relatively limited at present, and may rightly seem more distant to many, given the predominantly interpersonal and complex interventional nature of the specialty. In this overview, we will explore the current state and anticipated future impact of AI on Radiation Oncology, in detail, focusing on key topics from multiple stakeholder perspectives, as well as the role our specialty may play in helping to shape the future of AI within the larger spectrum of medicine.

SEGMENTATION AND CLASSIFICATION OF ISCHEMIC STROKE USING OPTIMIZED FEATURES IN BRAIN MRI

Detection of ischemic stroke using brain magnetic resonance imaging (MRI) images is vital and a challenging task in clinical practice. We propose a novel method based on optimization technique to identify stroke lesion in diffusion-weighted imaging (DWI) MRI sequences of the brain. The algorithm was tested in a specific slice having large area of stroke region from a series of 292 real-time images obtained from different stroke affected subjects from IMS and SUM Hospital. The proposed method consists of pre-processing, segmentation, extraction of important features and classification of stroke type. The particle swarm optimization (PSO) and Darwinian particle swarm optimization (DPSO) algorithms were applied in segmenting the stroke lesions. The important features were extracted with the gray-level co-occurrence matrix (GLCM) algorithm and in decision making process, the feature set is classified into three types of stroke according to The Oxfordshire Community Stroke Project (OCSP) classification using support vector machine (SVM) classifier. The lesion area was segmented effectively with DPSO process with classification weighted accuracy of 90.23%, which is higher than PSO method having weighted accuracy of 85.19%. Similarly, the values of different measured parameters were high in DPSO technique, the computational time was also higher in DPSO method for segmenting the stroke lesions. These results confirm that the DPSO-based approach with SVM classifier is an effective way to identify the decision making process of ischemic stroke lesion in MRI images of the brain.

Radiomics in Brain Tumor: Image Assessment, Quantitative Feature Descriptors, and Machine-Learning Approaches

Radiomics describes a broad set of computational methods that extract quantitative features from radiographic images. The resulting features can be used to inform imaging diagnosis, prognosis, and therapy response in oncology. However, major challenges remain for methodologic developments to optimize feature extraction and provide rapid information flow in clinical settings. Equally important, to be clinically useful, predictive radiomic properties must be clearly linked to meaningful biologic characteristics and qualitative imaging properties familiar to radiologists. Here we use a cross-disciplinary approach to highlight studies in radiomics. We review brain tumor radiologic studies (eg, imaging interpretation) through computational models (eg, computer vision and machine learning) that provide novel clinical insights. We outline current quantitative image feature extraction and prediction strategies with different levels of available clinical classes for supporting clinical decision-making. We further discuss machine-learning challenges and data opportunities to advance radiomic studies.

End-to-end workflow for finite element analysis of tumor treating fields in glioblastomas

Tumor Treating Fields (TTFields) therapy is an approved modality of treatment for glioblastoma. Patient anatomy-based finite element analysis (FEA) has the potential to reveal not only how these fields affect tumor control but also how to improve efficacy. While the automated tools for segmentation speed up the generation of FEA models, multi-step manual corrections are required, including removal of disconnected voxels, incorporation of unsegmented structures and the addition of 36 electrodes plus gel layers matching the TTFields transducers. Existing approaches are also not scalable for the high throughput analysis of large patient volumes. A semi-automated workflow was developed to prepare FEA models for TTFields mapping in the human brain. Magnetic resonance imaging (MRI) pre-processing, segmentation, electrode and gel placement, and post-processing were all automated. The material properties of each tissue were applied to their corresponding mask in silico using COMSOL Multiphysics (COMSOL, Burlington, MA, USA). The fidelity of the segmentations with and without post-processing was compared against the full semi-automated segmentation workflow approach using Dice coefficient analysis. The average relative differences for the electric fields generated by COMSOL were calculated in addition to observed differences in electric field-volume histograms. Furthermore, the mesh file formats in MPHTXT and NASTRAN were also compared using the differences in the electric field-volume histogram. The Dice coefficient was less for auto-segmentation without versus auto-segmentation with post-processing, indicating convergence on a manually corrected model. An existent but marginal relative difference of electric field maps from models with manual correction versus those without was identified, and a clear advantage of using the NASTRAN mesh file format was found. The software and workflow outlined in this article may be used to accelerate the investigation of TTFields in glioblastoma patients by facilitating the creation of FEA models derived from patient MRI datasets.

Rapid and Accurate MRI Segmentation of Peritumoral Brain Edema in Meningiomas

PURPOSE

The extent of peritumoral brain edema (PTBE) in meningiomas commonly affects the clinical outcome. Despite its importance, edema volume is usually highly inaccurately approximated to a spheroid shape. We tested the accuracy and the reproducibility of semiautomatic lesion management software for the analysis of PTBE in a homogeneous case series of surgically confirmed intracranial meningiomas.

METHODS

PTBE volume was calculated on magnetic resonance images in 50 patients with intracranial meningiomas using commercial lesion management software (Vue PACS Livewire, Carestream, Rochester, NY, USA). Inter and intraobserver agreement evaluation and a comparison between manual volume calculation, the semiautomatic software and spheroid approximation were performed in 22 randomly selected patients.

RESULTS

The calculation of edema volume was possible in all cases irrespective of the extent of the signal changes. The median time for each calculation was 3 min. Interobserver and intraobserver agreement confirmed the reproducibility of the method. Comparison with standard (fully manual) calculation confirmed the accuracy of this software.

CONCLUSIONS

Our study showed a high level of reproducibility of this semiautomatic computational method for peritumoral brain edema. It is rapid and easy to use after relatively short training and is suitable for implementation in clinical practice.

SEGMENTATION OF BRAIN MRIs BY SUPPORT VECTOR MACHINE: DETECTION AND CHARACTERIZATION OF STROKES

The aim of our work is the segmentation of healthy and pathological brains to obtain brain structures and extract strokes. We used real magnetic resonance (MR) images weighted on diffusion. The brain was isolated, and the images were filtered by an anisotropic filter, and then segmented by support vector machines (SVMs). We first applied the method on synthetic images to test the performance of the algorithm and adjust the parameters. Then, we compared our results with those obtained by a cooperative approach proposed in a previous paper.

Automated tumor volumetry using computer-aided image segmentation

RATIONALE AND OBJECTIVES

Accurate segmentation of brain tumors, and quantification of tumor volume, is important for diagnosis, monitoring, and planning therapeutic intervention. Manual segmentation is not widely used because of time constraints. Previous efforts have mainly produced methods that are tailored to a particular type of tumor or acquisition protocol and have mostly failed to produce a method that functions on different tumor types and is robust to changes in scanning parameters, resolution, and image quality, thereby limiting their clinical value. Herein, we present a semiautomatic method for tumor segmentation that is fast, accurate, and robust to a wide variation in image quality and resolution.

MATERIALS AND METHODS

A semiautomatic segmentation method based on the geodesic distance transform was developed and validated by using it to segment 54 brain tumors. Glioblastomas, meningiomas, and brain metastases were segmented. Qualitative validation was based on physician ratings provided by three clinical experts. Quantitative validation was based on comparing semiautomatic and manual segmentations.

RESULTS

Tumor segmentations obtained using manual and automatic methods were compared quantitatively using the Dice measure of overlap. Subjective evaluation was performed by having human experts rate the computerized segmentations on a 0-5 rating scale where 5 indicated perfect segmentation.

CONCLUSIONS

The proposed method addresses a significant, unmet need in the field of neuro-oncology. Specifically, this method enables clinicians to obtain accurate and reproducible tumor volumes without the need for manual segmentation.

Rough-fuzzy clustering and unsupervised feature selection for wavelet based MR image segmentation

Image segmentation is an indispensable process in the visualization of human tissues, particularly during clinical analysis of brain magnetic resonance (MR) images. For many human experts, manual segmentation is a difficult and time consuming task, which makes an automated brain MR image segmentation method desirable. In this regard, this paper presents a new segmentation method for brain MR images, integrating judiciously the merits of rough-fuzzy computing and multiresolution image analysis technique. The proposed method assumes that the major brain tissues, namely, gray matter, white matter, and cerebrospinal fluid from the MR images are considered to have different textural properties. The dyadic wavelet analysis is used to extract the scale-space feature vector for each pixel, while the rough-fuzzy clustering is used to address the uncertainty problem of brain MR image segmentation. An unsupervised feature selection method is introduced, based on maximum relevance-maximum significance criterion, to select relevant and significant textural features for segmentation problem, while the mathematical morphology based skull stripping preprocessing step is proposed to remove the non-cerebral tissues like skull. The performance of the proposed method, along with a comparison with related approaches, is demonstrated on a set of synthetic and real brain MR images using standard validity indices.

Iterative probabilistic voxel labeling: automated segmentation for analysis of The Cancer Imaging Archive glioblastoma images

BACKGROUND AND PURPOSE

Robust, automated segmentation algorithms are required for quantitative analysis of large imaging datasets. We developed an automated method that identifies and labels brain tumor-associated pathology by using an iterative probabilistic voxel labeling using k-nearest neighbor and Gaussian mixture model classification. Our purpose was to develop a segmentation method which could be applied to a variety of imaging from The Cancer Imaging Archive.

MATERIALS AND METHODS

Images from 2 sets of 15 randomly selected subjects with glioblastoma from The Cancer Imaging Archive were processed by using the automated algorithm. The algorithm-defined tumor volumes were compared with those segmented by trained operators by using the Dice similarity coefficient.

RESULTS

Compared with operator volumes, algorithm-generated segmentations yielded mean Dice similarities of 0.92 ± 0.03 for contrast-enhancing volumes and 0.84 ± 0.09 for FLAIR hyperintensity volumes. These values compared favorably with the means of Dice similarity coefficients between the operator-defined segmentations: 0.92 ± 0.03 for contrast-enhancing volumes and 0.92 ± 0.05 for FLAIR hyperintensity volumes. Robust segmentations can be achieved when only postcontrast T1WI and FLAIR images are available.

CONCLUSIONS

Iterative probabilistic voxel labeling defined tumor volumes that were highly consistent with operator-defined volumes. Application of this algorithm could facilitate quantitative assessment of neuroimaging from patients with glioblastoma for both research and clinical indications.

Brain tumor classification using the diffusion tensor image segmentation (D-SEG) technique

Background

There is an increasing demand for noninvasive brain tumor biomarkers to guide surgery and subsequent oncotherapy. We present a novel whole-brain diffusion tensor imaging (DTI) segmentation (D-SEG) to delineate tumor volumes of interest (VOIs) for subsequent classification of tumor type. D-SEG uses isotropic (p) and anisotropic (q) components of the diffusion tensor to segment regions with similar diffusion characteristics.

Methods

DTI scans were acquired from 95 patients with low- and high-grade glioma, metastases, and meningioma and from 29 healthy subjects. D-SEG uses k-means clustering of the 2D (p,q) space to generate segments with different isotropic and anisotropic diffusion characteristics.

Results

Our results are visualized using a novel RGB color scheme incorporating p, q and T2-weighted information within each segment. The volumetric contribution of each segment to gray matter, white matter, and cerebrospinal fluid spaces was used to generate healthy tissue D-SEG spectra. Tumor VOIs were extracted using a semiautomated flood-filling technique and D-SEG spectra were computed within the VOI. Classification of tumor type using D-SEG spectra was performed using support vector machines. D-SEG was computationally fast and stable and delineated regions of healthy tissue from tumor and edema. D-SEG spectra were consistent for each tumor type, with constituent diffusion characteristics potentially reflecting regional differences in tissue microstructure. Support vector machines classified tumor type with an overall accuracy of 94.7%, providing better classification than previously reported.

Conclusions

D-SEG presents a user-friendly, semiautomated biomarker that may provide a valuable adjunct in noninvasive brain tumor diagnosis and treatment planning.

Kernel Sparse Models for Automated Tumor Segmentation

In this paper, we propose sparse coding-based approaches for segmentation of tumor regions from magnetic resonance (MR) images. Sparse coding with data-adapted dictionaries has been successfully employed in several image recovery and vision problems. The proposed approaches obtain sparse codes for each pixel in brain MR images considering their intensity values and location information. Since it is trivial to obtain pixel-wise sparse codes, and combining multiple features in the sparse coding setup is not straight-forward, we propose to perform sparse coding in a high-dimensional feature space where non-linear similarities can be effectively modeled. We use the training data from expert-segmented images to obtain kernel dictionaries with the kernel K-lines clustering procedure. For a test image, sparse codes are computed with these kernel dictionaries, and they are used to identify the tumor regions. This approach is completely automated, and does not require user intervention to initialize the tumor regions in a test image. Furthermore, a low complexity segmentation approach based on kernel sparse codes, which allows the user to initialize the tumor region, is also presented. Results obtained with both the proposed approaches are validated against manual segmentation by an expert radiologist, and it is shown that proposed methods lead to accurate tumor identification.

Ensemble segmentation for GBM brain tumors on MR images using confidence-based averaging

PURPOSE

Ensemble segmentation methods combine the segmentation results of individual methods into a final one, with the goal of achieving greater robustness and accuracy. The goal of this study was to develop an ensemble segmentation framework for glioblastoma multiforme tumors on single-channel T1w postcontrast magnetic resonance images.

METHODS

Three base methods were evaluated in the framework: fuzzy connectedness, GrowCut, and voxel classification using support vector machine. A confidence map averaging (CMA) method was used as the ensemble rule.

RESULTS

The performance is evaluated on a comprehensive dataset of 46 cases including different tumor appearances. The accuracy of the segmentation result was evaluated using the F1-measure between the semiautomated segmentation result and the ground truth.

CONCLUSIONS

The results showed that the CMA ensemble result statistically approximates the best segmentation result of all the base methods for each case.

Semi-automatic segmentation of brain tumors using population and individual information

Efficient segmentation of tumors in medical images is of great practical importance in early diagnosis and radiation plan. This paper proposes a novel semi-automatic segmentation method based on population and individual statistical information to segment brain tumors in magnetic resonance (MR) images. First, high-dimensional image features are extracted. Neighborhood components analysis is proposed to learn two optimal distance metrics, which contain population and patient-specific information, respectively. The probability of each pixel belonging to the foreground (tumor) and the background is estimated by the k-nearest neighborhood classifier under the learned optimal distance metrics. A cost function for segmentation is constructed through these probabilities and is optimized using graph cuts. Finally, some morphological operations are performed to improve the achieved segmentation results. Our dataset consists of 137 brain MR images, including 68 for training and 69 for testing. The proposed method overcomes segmentation difficulties caused by the uneven gray level distribution of the tumors and even can get satisfactory results if the tumors have fuzzy edges. Experimental results demonstrate that the proposed method is robust to brain tumor segmentation.

Concurrent tumor segmentation and registration with uncertainty-based sparse non-uniform graphs

In this paper, we present a graph-based concurrent brain tumor segmentation and atlas to diseased patient registration framework. Both segmentation and registration problems are modeled using a unified pairwise discrete Markov Random Field model on a sparse grid superimposed to the image domain. Segmentation is addressed based on pattern classification techniques, while registration is performed by maximizing the similarity between volumes and is modular with respect to the matching criterion. The two problems are coupled by relaxing the registration term in the tumor area, corresponding to areas of high classification score and high dissimilarity between volumes. In order to overcome the main shortcomings of discrete approaches regarding appropriate sampling of the solution space as well as important memory requirements, content driven samplings of the discrete displacement set and the sparse grid are considered, based on the local segmentation and registration uncertainties recovered by the min marginal energies. State of the art results on a substantial low-grade glioma database demonstrate the potential of our method, while our proposed approach shows maintained performance and strongly reduced complexity of the model.

Radiologically defined ecological dynamics and clinical outcomes in glioblastoma multiforme: preliminary results

MATERIALS AND METHODS

We examined pretreatment magnetic resonance imaging (MRI) examinations from 32 patients with glioblastoma multiforme (GBM) enrolled in The Cancer Genome Atlas (TCGA). Spatial variations in T1 post-gadolinium and either T2-weighted or fluid attenuated inversion recovery sequences from each tumor MRI study were used to characterize each small region of the tumor by its local contrast enhancement and edema/cellularity ("habitat"). The patient cohort was divided into group 1 (survival < 400 days, n = 16) and group 2 (survival > 400 days, n = 16).

RESULTS

Histograms of relative values in each sequence demonstrated that the tumor regions were consistently divided into high and low blood contrast enhancement, each of which could be subdivided into regions of high, low, and intermediate cell density/interstitial edema. Group 1 tumors contained greater volumes of habitats with low contrast enhancement but intermediate and high cell density (not fully necrotic) than group 2. Both leave-one-out and 10-fold cross-validation schemes demonstrated that individual patients could be correctly assigned to the short or long survival group with 81.25% accuracy.

CONCLUSION

We demonstrate that novel image analytic techniques can characterize regional habitat variations in GBMs using combinations of MRI sequences. A preliminary study of 32 patients from the TCGA database found that the distribution of MRI-defined habitats varied significantly among the different survival groups. Radiologically defined ecological tumor analysis may provide valuable prognostic and predictive biomarkers in GBM and other tumors.

Segmentation, feature extraction, and multiclass brain tumor classification

Multiclass brain tumor classification is performed by using a diversified dataset of 428 post-contrast T1-weighted MR images from 55 patients. These images are of primary brain tumors namely astrocytoma (AS), glioblastoma multiforme (GBM), childhood tumor-medulloblastoma (MED), meningioma (MEN), secondary tumor-metastatic (MET), and normal regions (NR). Eight hundred fifty-six regions of interest (SROIs) are extracted by a content-based active contour model. Two hundred eighteen intensity and texture features are extracted from these SROIs. In this study, principal component analysis (PCA) is used for reduction of dimensionality of the feature space. These six classes are then classified by artificial neural network (ANN). Hence, this approach is named as PCA-ANN approach. Three sets of experiments have been performed. In the first experiment, classification accuracy by ANN approach is performed. In the second experiment, PCA-ANN approach with random sub-sampling has been used in which the SROIs from the same patient may get repeated during testing. It is observed that the classification accuracy has increased from 77 to 91 %. PCA-ANN has delivered high accuracy for each class: AS-90.74 %, GBM-88.46 %, MED-85 %, MEN-90.70 %, MET-96.67 %, and NR-93.78 %. In the third experiment, to remove bias and to test the robustness of the proposed system, data is partitioned in a manner such that the SROIs from the same patient are not common for training and testing sets. In this case also, the proposed system has performed well by delivering an overall accuracy of 85.23 %. The individual class accuracy for each class is: AS-86.15 %, GBM-65.1 %, MED-63.36 %, MEN-91.5 %, MET-65.21 %, and NR-93.3 %. A computer-aided diagnostic system comprising of developed methods for segmentation, feature extraction, and classification of brain tumors can be beneficial to radiologists for precise localization, diagnosis, and interpretation of brain tumors on MR images.