Local structure-based region-of-interest retrieval in brain MR images

RbQE: An Efficient Method for Content-Based Medical Image Retrieval Based on Query Expansion

Systems for retrieving and managing content-based medical images are becoming more important, especially as medical imaging technology advances and the medical image database grows. In addition, these systems can also use medical images to better grasp and gain a deeper understanding of the causes and treatments of different diseases, not just for diagnostic purposes. For achieving all these purposes, there is a critical need for an efficient and accurate content-based medical image retrieval (CBMIR) method. This paper proposes an efficient method (RbQE) for the retrieval of computed tomography (CT) and magnetic resonance (MR) images. RbQE is based on expanding the features of querying and exploiting the pre-trained learning models AlexNet and VGG-19 to extract compact, deep, and high-level features from medical images. There are two searching procedures in RbQE: a rapid search and a final search. In the rapid search, the original query is expanded by retrieving the top-ranked images from each class and is used to reformulate the query by calculating the mean values for deep features of the top-ranked images, resulting in a new query for each class. In the final search, the new query that is most similar to the original query will be used for retrieval from the database. The performance of the proposed method has been compared to state-of-the-art methods on four publicly available standard databases, namely, TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI. Experimental results show that the proposed method exceeds the compared methods by 0.84%, 4.86%, 1.24%, and 14.34% in average retrieval precision (ARP) for the TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI databases, respectively.

Biomedical image retrieval using adaptive neuro-fuzzy optimized classifier system

The quantity of scientific images associated with patient care has increased markedly in recent years due to the rapid development of hospitals and research facilities. Every hospital generates more medical photographs, resulting in more than 10 GB of data per day being produced by a single image appliance. Software is used extensively to scan and locate diagnostic photographs to identify patient's precise information, which can be valuable for medical science research and advancement. An image recovery system is used to meet this need. This paper suggests an optimized classifier framework focused on a hybrid adaptive neuro-fuzzy approach to accomplish this goal. In the user query, similarity measurement, and the image content, fuzzy sets represent the vagueness that occurs in such data sets. The optimized classifying method 'hybrid adaptive neuro-fuzzy is enhanced with the improved cuckoo search optimization. Score values are determined by utilizing the linear discriminant analysis (LDA) of such classified images. The preliminary findings indicate that the proposed approach can be more reliable and effective at estimation than can existing approaches.

Content based medical image retrieval based on new efficient local neighborhood wavelet feature descriptor

This paper presents a new class of local neighborhood based wavelet feature descriptor (LNWFD) for content based medical image retrieval (CBMIR). To retrieve images effectively from large medical databases is backbone of diagnosis. Existing wavelet transform based medical image retrieval methods suffer from high length feature vector with confined retrieval performance. Triplet half-band filter bank (THFB) enhanced the properties of wavelet filters using three kernels. The influence of THFB has employed in the proposed method. First, triplet half-band filter bank (THFB) is used for single level wavelet decomposition to obtain four sub-bands. Next, the relationship among wavelet coefficients is exploited at each sub-band using 3 × 3 neighborhood window to form LNWFD pattern. The novelty of the proposed descriptor lies in exploring relation between wavelet transform values of pixels rather than intensity values which gives more detail local information in wavelet sub-bands. Thus, proposed feature descriptor is robust against illumination. Manhattan distance is used to compute similarity between query feature vector and feature vector of database. The proposed method is tested for medical image retrieval using OASIS-MRI, NEMA-CT, and Emphysema-CT databases. The average retrieval precisions achieved are 71.45%, 99.51% of OASIS-MRI and NEMA-CT databases for top ten matches considered respectively and 55.51% of Emphysema-CT database for top 50 matches. The superiority in terms of performance of the proposed method is confirmed by the experimental results over the well-known existing descriptors.

Local Wavelet Pattern: A New Feature Descriptor for Image Retrieval in Medical CT Databases

A new image feature description based on the local wavelet pattern (LWP) is proposed in this paper to characterize the medical computer tomography (CT) images for content-based CT image retrieval. In the proposed work, the LWP is derived for each pixel of the CT image by utilizing the relationship of center pixel with the local neighboring information. In contrast to the local binary pattern that only considers the relationship between a center pixel and its neighboring pixels, the presented approach first utilizes the relationship among the neighboring pixels using local wavelet decomposition, and finally considers its relationship with the center pixel. A center pixel transformation scheme is introduced to match the range of center value with the range of local wavelet decomposed values. Moreover, the introduced local wavelet decomposition scheme is centrally symmetric and suitable for CT images. The novelty of this paper lies in the following two ways: 1) encoding local neighboring information with local wavelet decomposition and 2) computing LWP using local wavelet decomposed values and transformed center pixel values. We tested the performance of our method over three CT image databases in terms of the precision and recall. We also compared the proposed LWP descriptor with the other state-of-the-art local image descriptors, and the experimental results suggest that the proposed method outperforms other methods for CT image retrieval.

Content-based image retrieval for brain MRI: an image-searching engine and population-based analysis to utilize past clinical data for future diagnosis

Radiological diagnosis is based on subjective judgment by radiologists. The reasoning behind this process is difficult to document and share, which is a major obstacle in adopting evidence-based medicine in radiology. We report our attempt to use a comprehensive brain parcellation tool to systematically capture image features and use them to record, search, and evaluate anatomical phenotypes. Anatomical images (T1-weighted MRI) were converted to a standardized index by using a high-dimensional image transformation method followed by atlas-based parcellation of the entire brain. We investigated how the indexed anatomical data captured the anatomical features of healthy controls and a population with Primary Progressive Aphasia (PPA). PPA was chosen because patients have apparent atrophy at different degrees and locations, thus the automated quantitative results can be compared with trained clinicians' qualitative evaluations. We explored and tested the power of individual classifications and of performing a search for images with similar anatomical features in a database using partial least squares-discriminant analysis (PLS-DA) and principal component analysis (PCA). The agreement between the automated z-score and the averaged visual scores for atrophy (r = 0.8) was virtually the same as the inter-evaluator agreement. The PCA plot distribution correlated with the anatomical phenotypes and the PLS-DA resulted in a model with an accuracy of 88% for distinguishing PPA variants. The quantitative indices captured the main anatomical features. The indexing of image data has a potential to be an effective, comprehensive, and easily translatable tool for clinical practice, providing new opportunities to mine clinical databases for medical decision support.

•

Brain parcellation tools define structures automatically and convert images into standardized and quantitative matrices.

•

We tested if an automated tool and the resultant vector of structural volumes can accurately capture anatomical phenotypes.

•

The agreement between visual and automated atrophy detection was virtually the same as the inter-evaluator agreement.

•

The quantitative indices captured the main anatomical features in brains with atrophy in different degrees and location.

•

The image quantification has potential to be an effective, comprehensive, and easily translatable tool for clinical practice.

Brain CT image similarity retrieval method based on uncertain location graph

A number of brain computed tomography (CT) images stored in hospitals that contain valuable information should be shared to support computer-aided diagnosis systems. Finding the similar brain CT images from the brain CT image database can effectively help doctors diagnose based on the earlier cases. However, the similarity retrieval for brain CT images requires much higher accuracy than the general images. In this paper, a new model of uncertain location graph (ULG) is presented for brain CT image modeling and similarity retrieval. According to the characteristics of brain CT image, we propose a novel method to model brain CT image to ULG based on brain CT image texture. Then, a scheme for ULG similarity retrieval is introduced. Furthermore, an effective index structure is applied to reduce the searching time. Experimental results reveal that our method functions well on brain CT images similarity retrieval with higher accuracy and efficiency.

Multi-Channel neurodegenerative pattern analysis and its application in Alzheimer's disease characterization

Neuroimaging has played an important role in non-invasive diagnosis and differentiation of neurodegenerative disorders, such as Alzheimer's disease and Mild Cognitive Impairment. Various features have been extracted from the neuroimaging data to characterize the disorders, and these features can be roughly divided into global and local features. Recent studies show a tendency of using local features in disease characterization, since they are capable of identifying the subtle disease-specific patterns associated with the effects of the disease on human brain. However, problems arise if the neuroimaging database involved multiple disorders or progressive disorders, as disorders of different types or at different progressive stages might exhibit different degenerative patterns. It is difficult for the researchers to reach consensus on what brain regions could effectively distinguish multiple disorders or multiple progression stages. In this study we proposed a Multi-Channel pattern analysis approach to identify the most discriminative local brain metabolism features for neurodegenerative disorder characterization. We compared our method to global methods and other pattern analysis methods based on clinical expertise or statistics tests. The preliminary results suggested that the proposed Multi-Channel pattern analysis method outperformed other approaches in Alzheimer's disease characterization, and meanwhile provided important insights into the underlying pathology of Alzheimer's disease and Mild Cognitive Impairment.

Content-based medical image retrieval: a survey of applications to multidimensional and multimodality data

Medical imaging is fundamental to modern healthcare, and its widespread use has resulted in the creation of image databases, as well as picture archiving and communication systems. These repositories now contain images from a diverse range of modalities, multidimensional (three-dimensional or time-varying) images, as well as co-aligned multimodality images. These image collections offer the opportunity for evidence-based diagnosis, teaching, and research; for these applications, there is a requirement for appropriate methods to search the collections for images that have characteristics similar to the case(s) of interest. Content-based image retrieval (CBIR) is an image search technique that complements the conventional text-based retrieval of images by using visual features, such as color, texture, and shape, as search criteria. Medical CBIR is an established field of study that is beginning to realize promise when applied to multidimensional and multimodality medical data. In this paper, we present a review of state-of-the-art medical CBIR approaches in five main categories: two-dimensional image retrieval, retrieval of images with three or more dimensions, the use of nonimage data to enhance the retrieval, multimodality image retrieval, and retrieval from diverse datasets. We use these categories as a framework for discussing the state of the art, focusing on the characteristics and modalities of the information used during medical image retrieval.

Gross feature recognition of Anatomical Images based on Atlas grid (GAIA): Incorporating the local discrepancy between an atlas and a target image to capture the features of anatomic brain MRI

We aimed to develop a new method to convert T1-weighted brain MRIs to feature vectors, which could be used for content-based image retrieval (CBIR). To overcome the wide range of anatomical variability in clinical cases and the inconsistency of imaging protocols, we introduced the Gross feature recognition of Anatomical Images based on Atlas grid (GAIA), in which the local intensity alteration, caused by pathological (e.g., ischemia) or physiological (development and aging) intensity changes, as well as by atlas–image misregistration, is used to capture the anatomical features of target images.

As a proof-of-concept, the GAIA was applied for pattern recognition of the neuroanatomical features of multiple stages of Alzheimer's disease, Huntington's disease, spinocerebellar ataxia type 6, and four subtypes of primary progressive aphasia. For each of these diseases, feature vectors based on a training dataset were applied to a test dataset to evaluate the accuracy of pattern recognition. The feature vectors extracted from the training dataset agreed well with the known pathological hallmarks of the selected neurodegenerative diseases. Overall, discriminant scores of the test images accurately categorized these test images to the correct disease categories. Images without typical disease-related anatomical features were misclassified. The proposed method is a promising method for image feature extraction based on disease-related anatomical features, which should enable users to submit a patient image and search past clinical cases with similar anatomical phenotypes.

•

A novel method to convert anatomical brain MRIs to feature vectors is introduced.

•

Degree of local atlas–image disagreement is used to capture the anatomical features.

•

The method was applied for pattern recognition of various neurodegenerative diseases.

•

The feature vectors agreed well with the known pathological hallmarks of diseases.

•

The method accurately categorized test images to the correct disease categories.

Atlas-based neuroinformatics via MRI: harnessing information from past clinical cases and quantitative image analysis for patient care

With the ever-increasing amount of anatomical information radiologists have to evaluate for routine diagnoses, computational support that facilitates more efficient education and clinical decision making is highly desired. Despite the rapid progress of image analysis technologies for magnetic resonance imaging of the human brain, these methods have not been widely adopted for clinical diagnoses. To bring computational support into the clinical arena, we need to understand the decision-making process employed by well-trained clinicians and develop tools to simulate that process. In this review, we discuss the potential of atlas-based clinical neuroinformatics, which consists of annotated databases of anatomical measurements grouped according to their morphometric phenotypes and coupled with the clinical informatics upon which their diagnostic groupings are based. As these are indexed via parametric representations, we can use image retrieval tools to search for phenotypes along with their clinical metadata. The review covers the current technology, preliminary data, and future directions of this field.

Content-based retrieval of brain tumor in contrast-enhanced MRI images using tumor margin information and learned distance metric

PURPOSE

A content-based image retrieval (CBIR) method for T1-weighted contrast-enhanced MRI (CE-MRI) images of brain tumors is presented for diagnosis aid. The method is thoroughly evaluated on a large image dataset.

METHODS

Using the tumor region as a query, the authors' CBIR system attempts to retrieve tumors of the same pathological category. Aside from commonly used features such as intensity, texture, and shape features, the authors use a margin information descriptor (MID), which is capable of describing the characteristics of tissue surrounding a tumor, for representing image contents. In addition, the authors designed a distance metric learning algorithm called Maximum mean average Precision Projection (MPP) to maximize the smooth approximated mean average precision (mAP) to optimize retrieval performance.

RESULTS

The effectiveness of MID and MPP algorithms was evaluated using a brain CE-MRI dataset consisting of 3108 2D scans acquired from 235 patients with three categories of brain tumors (meningioma, glioma, and pituitary tumor). By combining MID and other features, the mAP of retrieval increased by more than 6% with the learned distance metrics. The distance metric learned by MPP significantly outperformed the other two existing distance metric learning methods in terms of mAP. The CBIR system using the proposed strategies achieved a mAP of 87.3% and a precision of 89.3% when top 10 images were returned by the system. Compared with scale-invariant feature transform, the MID, which uses the intensity profile as descriptor, achieves better retrieval performance.

CONCLUSIONS

Incorporating tumor margin information represented by MID with the distance metric learned by the MPP algorithm can substantially improve the retrieval performance for brain tumors in CE-MRI.

Histology image retrieval in optimised multi-feature spaces

Content based histology image retrieval systems have shown great potential in supporting decision making in clinical activities, teaching, and biological research. In content based image retrieval, feature combination plays a key role. It aims at enhancing the descriptive power of visual features corresponding to semantically meaningful queries. It is particularly valuable in histology image analysis where intelligent mechanisms are needed for interpreting varying tissue composition and architecture into histological concepts. This paper presents an approach to automatically combine heterogeneous visual features for histology image retrieval. The aim is to obtain the most representative fusion model for a particular keyword that is associated to multiple query images. The core of this approach is a multi-objective learning method, which aims to understand an optimal visual-semantic matching function by jointly considering the different preferences of the group of query images. The task is posed as an optimisation problem, and a multi-objective optimisation strategy is employed in order to handle potential contradictions in the query images associated to the same keyword. Experiments were performed on two different collections of histology images. The results show that it is possible to improve a system for content based histology image retrieval by using an appropriately defined multi-feature fusion model, which takes careful consideration of the structure and distribution of visual features.

Local tetra patterns: a new feature descriptor for content-based image retrieval

In this paper, we propose a novel image indexing and retrieval algorithm using local tetra patterns (LTrPs) for content-based image retrieval (CBIR). The standard local binary pattern (LBP) and local ternary pattern (LTP) encode the relationship between the referenced pixel and its surrounding neighbors by computing gray-level difference. The proposed method encodes the relationship between the referenced pixel and its neighbors, based on the directions that are calculated using the first-order derivatives in vertical and horizontal directions. In addition, we propose a generic strategy to compute nth-order LTrP using (n - 1)th-order horizontal and vertical derivatives for efficient CBIR and analyze the effectiveness of our proposed algorithm by combining it with the Gabor transform. The performance of the proposed method is compared with the LBP, the local derivative patterns, and the LTP based on the results obtained using benchmark image databases viz., Corel 1000 database (DB1), Brodatz texture database (DB2), and MIT VisTex database (DB3). Performance analysis shows that the proposed method improves the retrieval result from 70.34%/44.9% to 75.9%/48.7% in terms of average precision/average recall on database DB1, and from 79.97% to 85.30% and 82.23% to 90.02% in terms of average retrieval rate on databases DB2 and DB3, respectively, as compared with the standard LBP.

Semantic image retrieval in magnetic resonance brain volumes

Practitioners in the area of neurology often need to retrieve multimodal magnetic resonance (MR) images of the brain to study disease progression and to correlate observations across multiple subjects. In this paper, a novel technique for retrieving 2-D MR images (slices) in 3-D brain volumes is proposed. Given a 2-D MR query slice, the technique identifies the 3-D volume among multiple subjects in the database, associates the query slice with a specific region of the brain, and retrieves the matching slice within this region in the identified volumes. The proposed technique is capable of retrieving an image in multimodal and noisy scenarios. In this study, support vector machines (SVM) are used for identifying 3-D MR volume and for performing semantic classification of the human brain into various semantic regions. In order to achieve reliable image retrieval performance in the presence of misalignments, an image registration-based retrieval framework is developed. The proposed retrieval technique is tested on various modalities. The test results reveal superior robustness performance with respect to accuracy, speed, and multimodality.

Directional binary wavelet patterns for biomedical image indexing and retrieval

A new algorithm for medical image retrieval is presented in the paper. An 8-bit grayscale image is divided into eight binary bit-planes, and then binary wavelet transform (BWT) which is similar to the lifting scheme in real wavelet transform (RWT) is performed on each bitplane to extract the multi-resolution binary images. The local binary pattern (LBP) features are extracted from the resultant BWT sub-bands. Three experiments have been carried out for proving the effectiveness of the proposed algorithm. Out of which two are meant for medical image retrieval and one for face retrieval. It is further mentioned that the database considered for three experiments are OASIS magnetic resonance imaging (MRI) database, NEMA computer tomography (CT) database and PolyU-NIRFD face database. The results after investigation shows a significant improvement in terms of their evaluation measures as compared to LBP and LBP with Gabor transform.