Object recognition in brain CT-scans: knowledge-based fusion of data from multiple feature extractors

An Ensemble of Deep Learning Object Detection Models for Anatomical and Pathological Regions in Brain MRI

This paper proposes ensemble strategies for the deep learning object detection models carried out by combining the variants of a model and different models to enhance the anatomical and pathological object detection performance in brain MRI. In this study, with the help of the novel Gazi Brains 2020 dataset, five different anatomical parts and one pathological part that can be observed in brain MRI were identified, such as the region of interest, eye, optic nerves, lateral ventricles, third ventricle, and a whole tumor. Firstly, comprehensive benchmarking of the nine state-of-the-art object detection models was carried out to determine the capabilities of the models in detecting the anatomical and pathological parts. Then, four different ensemble strategies for nine object detectors were applied to boost the detection performance using the bounding box fusion technique. The ensemble of individual model variants increased the anatomical and pathological object detection performance by up to 10% in terms of the mean average precision (mAP). In addition, considering the class-based average precision (AP) value of the anatomical parts, an up to 18% AP improvement was achieved. Similarly, the ensemble strategy of the best different models outperformed the best individual model by 3.3% mAP. Additionally, while an up to 7% better FAUC, which is the area under the TPR vs. FPPI curve, was achieved on the Gazi Brains 2020 dataset, a 2% better FAUC score was obtained on the BraTS 2020 dataset. The proposed ensemble strategies were found to be much more efficient in finding the anatomical and pathological parts with a small number of anatomic objects, such as the optic nerve and third ventricle, and producing higher TPR values, especially at low FPPI values, compared to the best individual methods.

Fuzzy-based computational simulations of brain functions – preliminary concept

Research on the computational models of the brain constitutes an important part of the current challenges within computational neuroscience. The current results are not satisfying. Despite the continuous efforts of scientists and clinicians, it is hard to fully explain all the mechanisms of a brain function. Computational models of the brain based on fuzzy logic, including ordered fuzzy numbers, may constitute another breakthrough in the aforementioned area, offering a completing position to the current state of the art. The aim of this paper is to assess the extent to which possible opportunities concerning computational brain models based on fuzzy logic techniques may be exploited both in the area of theoretical and experimental computational neuroscience and in clinical applications, including our own concept. The proposed approach can open a family of novel methods for a more effective and (neuro)biologically reliable brain simulation based on fuzzy logic techniques useful in both basic sciences and applied sciences.

Image fusion of mass spectrometry and microscopy: a multimodality paradigm for molecular tissue mapping

A new predictive imaging modality is created through the ‘fusion’ of two distinct technologies: imaging mass spectrometry (IMS) and microscopy. IMS-generated molecular maps, rich in chemical information but having coarse spatial resolution, are combined with optical microscopy maps, which have relatively low chemical specificity but high spatial information. The resulting images combine the advantages of both technologies, enabling prediction of a molecular distribution both at high spatial resolution and with high chemical specificity. Multivariate regression is used to model variables in one technology, using variables from the other technology. Several applications demonstrate the remarkable potential of image fusion: (i) ‘sharpening’ of IMS images, which uses microscopy measurements to predict ion distributions at a spatial resolution that exceeds that of measured ion images by ten times or more; (ii) prediction of ion distributions in tissue areas that were not measured by IMS; and (iii) enrichment of biological signals and attenuation of instrumental artifacts, revealing insights that are not easily extracted from either microscopy or IMS separately. Image fusion enables a new multi-modality paradigm for tissue exploration whereby mining relationships between different imaging sensors yields novel imaging modalities that combine and surpass what can be gleaned from the individual technologies alone.

Automated medical image segmentation techniques

Accurate segmentation of medical images is a key step in contouring during radiotherapy planning. Computed topography (CT) and Magnetic resonance (MR) imaging are the most widely used radiographic techniques in diagnosis, clinical studies and treatment planning. This review provides details of automated segmentation methods, specifically discussed in the context of CT and MR images. The motive is to discuss the problems encountered in segmentation of CT and MR images, and the relative merits and limitations of methods currently available for segmentation of medical images.

Automated ventricular systems segmentation in brain CT images by combining low-level segmentation and high-level template matching

Background

Accurate analysis of CT brain scans is vital for diagnosis and treatment of Traumatic Brain Injuries (TBI). Automatic processing of these CT brain scans could speed up the decision making process, lower the cost of healthcare, and reduce the chance of human error. In this paper, we focus on automatic processing of CT brain images to segment and identify the ventricular systems. The segmentation of ventricles provides quantitative measures on the changes of ventricles in the brain that form vital diagnosis information.

Methods

First all CT slices are aligned by detecting the ideal midlines in all images. The initial estimation of the ideal midline of the brain is found based on skull symmetry and then the initial estimate is further refined using detected anatomical features. Then a two-step method is used for ventricle segmentation. First a low-level segmentation on each pixel is applied on the CT images. For this step, both Iterated Conditional Mode (ICM) and Maximum A Posteriori Spatial Probability (MASP) are evaluated and compared. The second step applies template matching algorithm to identify objects in the initial low-level segmentation as ventricles. Experiments for ventricle segmentation are conducted using a relatively large CT dataset containing mild and severe TBI cases.

Results

Experiments show that the acceptable rate of the ideal midline detection is over 95%. Two measurements are defined to evaluate ventricle recognition results. The first measure is a sensitivity-like measure and the second is a false positive-like measure. For the first measurement, the rate is 100% indicating that all ventricles are identified in all slices. The false positives-like measurement is 8.59%. We also point out the similarities and differences between ICM and MASP algorithms through both mathematically relationships and segmentation results on CT images.

Conclusion

The experiments show the reliability of the proposed algorithms. The novelty of the proposed method lies in its incorporation of anatomical features for ideal midline detection and the two-step ventricle segmentation method. Our method offers the following improvements over existing approaches: accurate detection of the ideal midline and accurate recognition of ventricles using both anatomical features and spatial templates derived from Magnetic Resonance Images.

Segmentation framework based on label field fusion

In this paper, we put forward a novel fusion framework that mixes together label fields instead of observation data as is usually the case. Our framework takes as input two label fields: a quickly estimated and to-be-refined segmentation map and a spatial region map that exhibits the shape of the main objects of the scene. These two label fields are fused together with a global energy function that is minimized with a deterministic iterative conditional mode algorithm. As explained in the paper, the energy function may implement a pure fusion strategy or a fusion-reaction function. In the latter case, a data-related term is used to make the optimization problem well posed. We believe that the conceptual simplicity, the small number of parameters, the use of a simple and fast deterministic optimizer that admits a natural implementation on a parallel architecture are among the main advantages of our approach. Our fusion framework is adapted to various computer vision applications among which are motion segmentation, motion estimation and occlusion detection.

Expert knowledge-guided segmentation system for brain MRI

We describe an automated 3-D segmentation system for in vivo brain magnetic resonance images (MRI). Our segmentation method combines a variety of filtering, segmentation, and registration techniques and makes maximum use of the available a priori biomedical expertise, both in an implicit and an explicit form. We approach the issue of boundary finding as a process of fitting a group of deformable templates (simplex mesh surfaces) to the contours of the target structures. These templates evolve in parallel, supervised by a series of rules derived from analyzing the template's dynamics and from medical experience. The templates are also constrained by knowledge on the expected textural and shape properties of the target structures. We apply our system to segment four brain structures (corpus callosum, ventricles, hippocampus, and caudate nuclei) and discuss its robustness to imaging characteristics and acquisition noise.

Identifying multiple abdominal organs from CT image series using a multimodule contextual neural network and spatial fuzzy rules

Identifying abdominal organs is one of the essential steps in visualizing organ structure to assist in teaching, clinical training, diagnosis, and medical image retrieval. However, due to partial volume effects, gray-level similarities of adjacent organs, contrast media affect, and the relatively high variations of organ position and shape, automatically identifying abdominal organs has always been a high challenging task. To conquer these difficulties, this paper proposes combining a multimodule contextual neural network and spatial fuzzy rules and fuzzy descriptors for automatically identifying abdominal organs from a series of CT image slices. The multimodule contextual neural network segments each image slice through a divide-and-conquer concept, embedded within multiple neural network modules, where the results obtained from each module are forwarded to other modules for integration, in which contextual constraints are enforced. With this approach, the difficulties arising from partial volume effects, gray-level similarities of adjacent organs, and contrast media affect can be reduced to the extreme. To address the issue of high variations in organ position and shape, spatial fuzzy rules and fuzzy descriptors are adopted, along with a contour modification scheme implementing consecutive organ region overlap constraints. This approach has been tested on 40 sets of abdominal CT images, where each set consists of about 40 image slices. We have found that 99% of the organ regions in the test images are correctly identified as its belonging organs, implying the high promise of the proposed method.

A knowledge-based approach to automatic detection of the spinal cord in CT images

Accurate planning of radiation therapy entails the definition of treatment volumes and a clear delimitation of normal tissue of which unnecessary exposure should be prevented. The spinal cord is a radiosensitive organ, which should be precisely identified because an overexposure to radiation may lead to undesired complications for the patient such as neuronal disfunction or paralysis. In this paper, a knowledge-based approach to identifying the spinal cord in computed tomography images of the thorax is presented. The approach relies on a knowledge-base which consists of a so-called anatomical structures map (ASM) and a task-oriented architecture called the plan solver. The ASM contains a frame-like knowledge representation of the macro-anatomy in the human thorax. The plan solver is responsible for determining the position, orientation and size of the structures of interest to radiation therapy. The plan solver relies on a number of image processing operators. Some are so-called atomic (e.g., thresholding and snakes) whereas others are composite. The whole system has been implemented on a standard PC. Experiments performed on the image material from 23 patients show that the approach results in a reliable recognition of the spinal cord (92% accuracy) and the spinal canal (85% accuracy). The lamina is more problematic to locate correctly (accuracy 72%). The position of the outer thorax is always determined correctly.

Medical image segmentation with knowledge-guided robust active contours

Medical image segmentation techniques typically require some form of expert human supervision to provide accurate and consistent identification of anatomic structures of interest. A novel segmentation technique was developed that combines a knowledge-based segmentation system with a sophisticated active contour model. This approach exploits the guidance of a higher-level process to robustly perform the segmentation of various anatomic structures. The user need not provide initial contour placement, and the high-level process carries out the required parameter optimization automatically. Knowledge about the anatomic structures to be segmented is defined statistically in terms of probability density functions of parameters such as location, size, and image intensity (eg, computed tomographic [CT] attenuation value). Preliminary results suggest that the performance of the algorithm at chest and abdominal CT is comparable to that of more traditional segmentation techniques like region growing and morphologic operators. In some cases, the active contour-based technique may outperform standard segmentation methods due to its capacity to fully enforce the available a priori knowledge concerning the anatomic structure of interest. The active contour algorithm is particularly suitable for integration with high-level image understanding frameworks, providing a robust and easily controlled low-level segmentation tool. Further study is required to determine whether the proposed algorithm is indeed capable of providing consistently superior segmentation.

Automatic segmentation of subcortical brain structures in MR images using information fusion

This paper reports a new automated method for the segmentation of internal cerebral structures using an information fusion technique. The information is provided both by images and expert knowledge, and consists in morphological, topological, and tissue constitution data. All this ambiguous, complementary and redundant information is managed using a three-step fusion scheme based on fuzzy logic. The information is first modeled into a common theoretical frame managing its imprecision and incertitude. The models are then fused and a decision is taken in order to reduce the imprecision and to increase the certainty in the location of the structures. The whole process is illustrated on the segmentation of thalamus, putamen, and head of the caudate nucleus from expert knowledge and magnetic resonance images, in a protocol involving 14 healthy volunteers. The quantitative validation is achieved by comparing computed, manually segmented structures and published data by means of indexes assessing the accuracy of volume estimation and spatial location. Results suggest a consistent volume estimation with respect to the expert quantification and published data, and a high spatial similarity of the segmented and computed structures. This method is generic and applicable to any structure that can be defined by expert knowledge and morphological images.

Segmentation and interpretation of MR brain images: an improved active shape model

This paper reports a novel method for fully automated segmentation that is based on description of shape and its variation using point distribution models (PDM's). An improvement of the active shape procedure introduced by Cootes and Taylor to find new examples of previously learned shapes using PDM's is presented. The new method for segmentation and interpretation of deep neuroanatomic structures such as thalamus, putamen, ventricular system, etc. incorporates a priori knowledge about shapes of the neuroanatomic structures to provide their robust segmentation and labeling in magnetic resonance (MR) brain images. The method was trained in eight MR brain images and tested in 19 brain images by comparison to observer-defined independent standards. Neuroanatomic structures in all testing images were successfully identified. Computer-identified and observer-defined neuroanatomic structures agreed well. The average labeling error was 7%+/-3%. Border positioning errors were quite small, with the average border positioning error of 0.8+/-0.1 pixels in 256 x 256 MR images. The presented method was specifically developed for segmentation of neuroanatomic structures in MR brain images. However, it is generally applicable to virtually any task involving deformable shape analysis.

Knowledge-based method for segmentation and analysis of lung boundaries in chest X-ray images

We present a knowledge-based approach to segmentation and analysis of the lung boundaries in chest X-rays. Image edges are matched to an anatomical model of the lung boundary using parametric features. A modular system architecture was developed which incorporates the model, image processing routines, an inference engine and a blackboard. Edges associated with the lung boundary are automatically identified and abnormal features are reported. In preliminary testing on 14 images for a set of 18 detectable abnormalities, the system showed a sensitivity of 88% and a specificity of 95% when compared with assessment by an experienced radiologist.

Fully automatic segmentation of the brain in MRI

A robust fully automatic method for segmenting the brain from head magnetic resonance (MR) images has been developed, which works even in the presence of radio frequency (RF) inhomogeneities. It has been successful in segmenting the brain in every slice from head images acquired from several different MRI scanners, using different-resolution images and different echo sequences. The method uses an integrated approach which employs image processing techniques based on anisotropic filters and "snakes" contouring techniques, and a priori knowledge, which is used to remove the eyes, which are tricky to remove based on image intensity alone. It is a multistage process, involving first removal of the background noise leaving a head mask, then finding a rough outline of the brain, then refinement of the rough brain outline to a final mask. The paper describes the main features of the method, and gives results for some brain studies.

Method for segmenting chest CT image data using an anatomical model: preliminary results

We present an automated, knowledge-based method for segmenting chest computed tomography (CT) datasets. Anatomical knowledge including expected volume, shape, relative position, and X-ray attenuation of organs provides feature constraints that guide the segmentation process. Knowledge is represented at a high level using an explicit anatomical model. The model is stored in a frame-based semantic network and anatomical variability is incorporated using fuzzy sets. A blackboard architecture permits the data representation and processing algorithms in the model domain to be independent of those in the image domain. Knowledge-constrained segmentation routines extract contiguous three-dimensional (3-D) sets of voxels, and their feature-space representations are posted on the blackboard. An inference engine uses fuzzy logic to match image to model objects based on the feature constraints. Strict separation of model and image domains allows for systematic extension of the knowledge base. In preliminary experiments, the method has been applied to a small number of thoracic CT datasets. Based on subjective visual assessment by experienced thoracic radiologists, basic anatomic structures such as the lungs, central tracheobronchial tree, chest wall, and mediastinum were successfully segmented. To demonstrate the extensibility of the system, knowledge was added to represent the more complex anatomy of lung lesions in contact with vessels or the chest wall. Visual inspection of these segmented lesions was also favorable. These preliminary results suggest that use of expert knowledge provides an increased level of automation compared with low-level segmentation techniques. Moreover, the knowledge-based approach may better discriminate between structures of similar attenuation and anatomic contiguity. Further validation is required.