Computer-Aided Diagnosis in Mammography Using Content-based Image Retrieval Approaches: Current Status and Future Perspectives

Evaluation of a Novel Content-Based Image Retrieval System for the Differentiation of Interstitial Lung Diseases in CT Examinations

To evaluate the reader's diagnostic performance against the ground truth with and without the help of a novel content-based image retrieval system (CBIR) that retrieves images with similar CT patterns from a database of 79 different interstitial lung diseases. We evaluated three novice readers' and three resident physicians' (with at least three years of experience) diagnostic performance evaluating 50 different CTs featuring 10 different patterns (e.g., honeycombing, tree-in bud, ground glass, bronchiectasis, etc.) and 24 different diseases (sarcoidosis, UIP, NSIP, Aspergillosis, COVID-19 pneumonia etc.). The participants read the cases first without assistance (and without feedback regarding correctness), and with a 2-month interval in a random order with the assistance of the novel CBIR. To invoke the CBIR, a ROI is placed into the pathologic pattern by the reader and the system retrieves diseases with similar patterns. To further narrow the differential diagnosis, the readers can consult an integrated textbook and have the possibility of selecting high-level semantic features representing clinical information (chronic, infectious, smoking status, etc.). We analyzed readers' accuracy without and with CBIR assistance and further tested the hypothesis that the CBIR would help to improve diagnostic performance utilizing Wilcoxon signed rank test. The novice readers demonstrated an unassisted accuracy of 18/28/44%, and an assisted accuracy of 84/82/90%, respectively. The resident physicians demonstrated an unassisted accuracy of 56/56/70%, and an assisted accuracy of 94/90/96%, respectively. For each reader, as well as overall, Sign test demonstrated statistically significant (p < 0.01) difference between the unassisted and the assisted reads. For students and physicians, Chi²-test and Mann-Whitney-U test demonstrated statistically significant (p < 0.01) difference for unassisted reads and statistically insignificant (p > 0.01) difference for assisted reads. The evaluated CBIR relying on pattern analysis and featuring the option to filter the results of the CBIR by predominant characteristics of the diseases via selecting high-level semantic features helped to drastically improve novices' and resident physicians' accuracy in diagnosing interstitial lung diseases in CT.

Analysis of human brain by magnetic resonance imaging using content-based image retrieval

OBJECTIVE

Content-based image retrieval (CBIR) is the most suitable and alternative method for older text searches that use keywords. This article aims to improve feature extraction as well as matching techniques designed for more accurate and precise CBIR systems, especially for brain scan images associated with various brain diseases and abnormalities. Tests should be described at an appropriate success rate.

METHODS

Various methods of producing medical images are discussed, and examples of biological applications are given. The discussion emphasizes as an introduction to CBIR the new method of echo-planar imaging, which is fully described. We have done here many methods related to digital image processing and we had developed a code for retrieving everything automatically. This application has been developed in Matlab software.

RESULTS

Testing the correctness and effectiveness of the system evolved becomes more important when the system is going to be used in real-time and more when it is for humankind, i.e., medical diagnosis. Nowadays, our science and technology areas as develop as we can say that we have such advanced medical equipment so that our thought and program can be capable that it is giving us useful results. Determining if whether the two images are identical or not, it depends on the point of view of the person.

CONCLUSIONS

In this paper, the outcome of feature extraction and matching by setting cutoff limit and threshold is pretty promising. Further studies can be done apart from computed tomography scans for a more generalized CBIR system.

Location of mammograms ROI's and reduction of false-positive

BACKGROUND AND OBJECTIVE

There are many work related with segmentation techniques, including nearest neighbor algorithm, fuzzy rules, morphological filters, image entropy, thresholding, machine learning, wavelet analysis, and so on. Such methods carry out the segmentation, but take a lot of processing time by modifying the content of the image or showing discern problems in homogeneous areas, and the segmentation technique is designed to work efficiently only with the techniques used. In this paper a method to segment mammograms in order to separate breast area from pectoral-muscle avoiding bright areas that produce noise and therefore reducing false-positives is presented.

METHODS

The proposed methodology is divided into four sections: 1) Pre-processing to acquire image and decreasing its size. 2) Improving the image quality through image thresholding and histogram equalization. 3) Localization of regions of interest (ROI) applying Scale-Invariant Feature Transform to find image's descriptors. Clustering methods were implemented to determine the best number of clusters and which of these represent the most significant breast area. Then found ROI's coordinates are compared with the position of abnormalities diagnosed by the Mammographic Image Analysis Society. 4) Microcalcifications (mcc) detection; wavelet transform is used, and to enhance its performance different high-pass filters and high-frequency emphasis filters are evaluated. Symlet wavelets: Sym8 and Sym16 were used with different decomposition level; images results from both processes are compared and only those elements in common are detected as microcalcifications.

RESULTS

Moreover, muscle's remnants in the corners of the regions of interest were removed using fuzzy c-means clustering. The best results in terms of sensitivity (91.27), false-positives per image (80.25), and precision (74.38) are compared with previous work.

CONCLUSIONS

Results shows that the breast area can be discriminated from the pectoral-muscle by avoiding to work with brightness areas that produces false positives. Moreover, because the image size is reduced the computer processing time will be decreased. This segmentation stage can be an addition to mammograms analysis broadly, not only to find mcc but abnormalities such as circumscribed masses, speculated masses and architectural distortion. Also is useful to create automatically an unsupervised segmentation in mammograms without stage of training.

A new approach to develop computer-aided diagnosis scheme of breast mass classification using deep learning technology

PURPOSE

To develop and test a deep learning based computer-aided diagnosis (CAD) scheme of mammograms for classifying between malignant and benign masses.

METHODS

An image dataset involving 560 regions of interest (ROIs) extracted from digital mammograms was used. After down-sampling each ROI from 512×512 to 64×64 pixel size, we applied an 8 layer deep learning network that involves 3 pairs of convolution-max-pooling layers for automatic feature extraction and a multiple layer perceptron (MLP) classifier for feature categorization to process ROIs. The 3 pairs of convolution layers contain 20, 10, and 5 feature maps, respectively. Each convolution layer is connected with a max-pooling layer to improve the feature robustness. The output of the sixth layer is fully connected with a MLP classifier, which is composed of one hidden layer and one logistic regression layer. The network then generates a classification score to predict the likelihood of ROI depicting a malignant mass. A four-fold cross validation method was applied to train and test this deep learning network.

RESULTS

The results revealed that this CAD scheme yields an area under the receiver operation characteristic curve (AUC) of 0.696±0.044, 0.802±0.037, 0.836±0.036, and 0.822±0.035 for fold 1 to 4 testing datasets, respectively. The overall AUC of the entire dataset is 0.790±0.019.

CONCLUSIONS

This study demonstrates the feasibility of applying a deep learning based CAD scheme to classify between malignant and benign breast masses without a lesion segmentation, image feature computation and selection process.

Assessment of performance and reproducibility of applying a content-based image retrieval scheme for classification of breast lesions

PURPOSE

To develop a new computer-aided diagnosis (CAD) scheme using a content-based image retrieval (CBIR) approach for classification between the malignant and benign breast lesions depicted on the digital mammograms and assess CAD performance and reproducibility.

METHODS

An image dataset including 820 regions of interest (ROIs) was used. Among them, 431 ROIs depict malignant lesions and 389 depict benign lesions. After applying an image preprocessing process to define the lesion center, two image features were computed from each ROI. The first feature is an average pixel value of a mapped region generated using a watershed algorithm. The second feature is an average pixel value difference between a ROI's center region and the rest of the image. A two-step CBIR approach uses these two features sequentially to search for ten most similar reference ROIs for each queried ROI. A similarity based classification score was then computed to predict the likelihood of the queried ROI depicting a malignant lesion. To assess the reproducibility of the CAD scheme, we selected another independent testing dataset of 100 ROIs. For each ROI in the testing dataset, we added four randomly queried lesion center pixels and examined the variation of the classification scores.

RESULTS

The area under the ROC curve (AUC) = 0.962 ± 0.006 was obtained when applying a leave-one-out validation method to 820 ROIs. Using the independent testing dataset, the initial AUC value was 0.832 ± 0.040, and using the median classification score of each ROI with five queried seeds, AUC value increased to 0.878 ± 0.035.

CONCLUSIONS

The authors demonstrated that (1) a simple and efficient CBIR scheme using two lesion density distribution related features achieved high performance in classifying breast lesions without actual lesion segmentation and (2) similar to the conventional CAD schemes using global optimization approaches, improving reproducibility is also one of the challenges in developing CAD schemes using a CBIR based regional optimization approach.

A CAD System for Hemorrhagic Stroke

Computer-aided detection/diagnosis (CAD) is a key component of routine clinical practice, increasingly used for detection, interpretation, quantification and decision support. Despite a critical need, there is no clinically accepted CAD system for stroke yet. Here we introduce a CAD system for hemorrhagic stroke. This CAD system segments, quantifies, and displays hematoma in 2D/3D, and supports evacuation of hemorrhage by thrombolytic treatment monitoring progression and quantifying clot removal. It supports seven-step workflow: select patient, add a new study, process patient's scans, show segmentation results, plot hematoma volumes, show 3D synchronized time series hematomas, and generate report. The system architecture contains four components: library, tools, application with user interface, and hematoma segmentation algorithm. The tools include a contour editor, 3D surface modeler, 3D volume measure, histogramming, hematoma volume plot, and 3D synchronized time-series hematoma display. The CAD system has been designed and implemented in C++. It has also been employed in the CLEAR and MISTIE phase-III, multicenter clinical trials. This stroke CAD system is potentially useful in research and clinical applications, particularly for clinical trials.

Identification of suitable fundus images using automated quality assessment methods

Retinal image quality assessment (IQA) is a crucial process for automated retinal image analysis systems to obtain an accurate and successful diagnosis of retinal diseases. Consequently, the first step in a good retinal image analysis system is measuring the quality of the input image. We present an approach for finding medically suitable retinal images for retinal diagnosis. We used a three-class grading system that consists of good, bad, and outlier classes. We created a retinal image quality dataset with a total of 216 consecutive images called the Diabetic Retinopathy Image Database. We identified the suitable images within the good images for automatic retinal image analysis systems using a novel method. Subsequently, we evaluated our retinal image suitability approach using the Digital Retinal Images for Vessel Extraction and Standard Diabetic Retinopathy Database Calibration level 1 public datasets. The results were measured through the F1 metric, which is a harmonic mean of precision and recall metrics. The highest F1 scores of the IQA tests were 99.60%, 96.50%, and 85.00% for good, bad, and outlier classes, respectively. Additionally, the accuracy of our suitable image detection approach was 98.08%. Our approach can be integrated into any automatic retinal analysis system with sufficient performance scores.

Quality assurance and training procedures for computer-aided detection and diagnosis systems in clinical use

Computer-aided detection/diagnosis (CAD) is increasingly used for decision support by clinicians for detection and interpretation of diseases. However, there are no quality assurance (QA) requirements for CAD in clinical use at present. QA of CAD is important so that end users can be made aware of changes in CAD performance both due to intentional or unintentional causes. In addition, end-user training is critical to prevent improper use of CAD, which could potentially result in lower overall clinical performance. Research on QA of CAD and user training are limited to date. The purpose of this paper is to bring attention to these issues, inform the readers of the opinions of the members of the American Association of Physicists in Medicine (AAPM) CAD subcommittee, and thus stimulate further discussion in the CAD community on these topics. The recommendations in this paper are intended to be work items for AAPM task groups that will be formed to address QA and user training issues on CAD in the future. The work items may serve as a framework for the discussion and eventual design of detailed QA and training procedures for physicists and users of CAD. Some of the recommendations are considered by the subcommittee to be reasonably easy and practical and can be implemented immediately by the end users; others are considered to be "best practice" approaches, which may require significant effort, additional tools, and proper training to implement. The eventual standardization of the requirements of QA procedures for CAD will have to be determined through consensus from members of the CAD community, and user training may require support of professional societies. It is expected that high-quality CAD and proper use of CAD could allow these systems to achieve their true potential, thus benefiting both the patients and the clinicians, and may bring about more widespread clinical use of CAD for many other diseases and applications. It is hoped that the awareness of the need for appropriate CAD QA and user training will stimulate new ideas and approaches for implementing such procedures efficiently and effectively as well as funding opportunities to fulfill such critical efforts.

Thyroid nodule recognition based on feature selection and pixel classification methods

Statistical approach is a valuable way to describe texture primitives. The aim of this study is to design and implement a classifier framework to automatically identify the thyroid nodules from ultrasound images. Using rigorous mathematical foundations, this article focuses on developing a discriminative texture analysis method based on texture variations corresponding to four biological areas (normal thyroid, thyroid nodule, subcutaneous tissues, and trachea). Our research follows three steps: automatic extraction of the most discriminative first-order statistical texture features, building a classifier that automatically optimizes and selects the valuable features, and correlating significant texture parameters with the four biological areas of interest based on pixel classification and location characteristics. Twenty ultrasound images of normal thyroid and 20 that present thyroid nodules were used. The analysis involves both the whole thyroid ultrasound images and the region of interests (ROIs). The proposed system and the classification results are validated using the receiver operating characteristics which give a better overall view of the classification performance of methods. It is found that the proposed approach is capable of identifying thyroid nodules with a correct classification rate of 83 % when whole image is analyzed and with a percent of 91 % when the ROIs are analyzed.

Regularization in retrieval-driven classification of clustered microcalcifications for breast cancer

We propose a regularization based approach for case-adaptive classification in computer-aided diagnosis (CAD) of breast cancer. The goal is to improve the classification accuracy on a query case by making use of a set of similar cases retrieved from an existing library of known cases. In the proposed approach, a prior is first derived from a traditional CAD classifier (which is typically pre-trained offline on a set of training cases). It is then used together with the retrieved similar cases to obtain an adaptive classifier on the query case. We consider two different forms for the regularization prior: one is fixed for all query cases and the other is allowed to vary with different query cases. In the experiments the proposed approach is demonstrated on a dataset of 1,006 clinical cases. The results show that it could achieve significant improvement in numerical efficiency compared with a previously proposed case adaptive approach (by about an order of magnitude) while maintaining similar (or better) improvement in classification accuracy; it could also adapt faster in performance with a small number of retrieved cases. Measured by the area of under the ROC curve (AUC), the regularization based approach achieved AUC = 0.8215, compared with AUC = 0.7329 for the baseline classifier (P-value = 0.001).

Towards a repository for standardized medical image and signal case data annotated with ground truth

Validation of medical signal and image processing systems requires quality-assured, representative and generally acknowledged databases accompanied by appropriate reference (ground truth) and clinical metadata, which are composed laboriously for each project and are not shared with the scientific community. In our vision, such data will be stored centrally in an open repository. We propose an architecture for a standardized case data and ground truth information repository supporting the evaluation and analysis of computer-aided diagnosis based on (a) the Reference Model for an Open Archival Information System (OAIS) provided by the NASA Consultative Committee for Space Data Systems (ISO 14721:2003), (b) the Dublin Core Metadata Initiative (DCMI) Element Set (ISO 15836:2009), (c) the Open Archive Initiative (OAI) Protocol for Metadata Harvesting, and (d) the Image Retrieval in Medical Applications (IRMA) framework. In our implementation, a portal bunches all of the functionalities that are needed for data submission and retrieval. The complete life cycle of the data (define, create, store, sustain, share, use, and improve) is managed. Sophisticated search tools make it easier to use the datasets, which may be merged from different providers. An integrated history record guarantees reproducibility. A standardized creation report is generated with a permanent digital object identifier. This creation report must be referenced by all of the data users. Peer-reviewed e-publishing of these reports will create a reputation for the data contributors and will form de-facto standards regarding image and signal datasets. Good practice guidelines for validation methodology complement the concept of the case repository. This procedure will increase the comparability of evaluation studies for medical signal and image processing methods and applications.

Retrieval boosted computer-aided diagnosis of clustered microcalcifications for breast cancer

PURPOSE

The authors propose an image-retrieval based approach for case-adaptive classifier design in computer-aided diagnosis (CADx). The conventional approach in CADx is to first train a pattern-classifier based on a set of existing training samples and then apply this classifier to subsequent new cases. The purpose of this work is to improve the classification accuracy of a CADx classifier by making use of a set of known cases retrieved from a reference library that are similar to the case under consideration.

METHODS

In the proposed approach, the authors will first apply image-retrieval to obtain a set of lesion images from a library of known cases that have similar image features to a case being diagnosed (i.e., query). These retrieved cases are then used to optimize a pattern-classifier toward boosting its classification accuracy on the query case. The basic idea is to put more emphasis on those cases that are similar to the query. The proposed approach is demonstrated first using a linear classifier and then extended to a nonlinear classifier induced by kernel principal component analysis.

RESULTS

The proposed retrieval-driven approach was tested on a library of mammogram images from 1006 cases (646 benign and 360 malignant) obtained from multiple institutions and was demonstrated to yield significant improvement in classification performance. Measured by the area under the receiver operating characteristic curve (AUC), the case-adaptive approach could boost the classification performance of a linear classifier from AUC = 0.7415 to AUC = 0.7807; similar improvement was also obtained for a nonlinear classifier, with AUC boosted from 0.7527 to 0.7838.

CONCLUSIONS

Use of additional cases from a reference library that have similar image features can improve the classification accuracy of a CADx classifier on a query case. It can even outperform retraining the classifier with all the cases from the entire reference library. This implies that cases with similar image features are more relevant in defining the local decision boundary of the CADx classifier around the query.

Assessment of performance and reliability of computer-aided detection scheme using content-based image retrieval approach and limited reference database

Content-based image retrieval approach was used in our computer-aided detection (CAD) schemes for breast cancer detection with mammography. In this study, we assessed CAD performance and reliability using a reference database including 1500 positive (breast mass) regions of interest (ROIs) and 1500 normal ROIs. To test the relationship between CAD performance and the similarity level between the queried ROI and the retrieved ROIs, we applied a set of similarity thresholds to the retrieved similar ROIs selected by the CAD schemes for all queried suspicious regions, and used only the ROIs that were above the threshold for assessing CAD performance at each threshold level. Using the leave-one-out testing method, we computed areas under receiver operating characteristic (ROC) curves (A(Z)) to assess CAD performance. The experimental results showed that as threshold increase, (1) less true positive ROIs can be referenced in the database than normal ROIs and (2) the A(Z) value was monotonically increased from 0.854 ± 0.004 to 0.932 ± 0.016. This study suggests that (1) in order to more accurately detect and diagnose subtle masses, a large and diverse database is required, and (2) assessing the reliability of the decision scores based on the similarity measurement is important in application of the CBIR-based CAD schemes when the limited database is used.

Exploring the potential of context-sensitive CADe in screening mammography

PURPOSE

Conventional computer-assisted detection (CADe) systems in screening mammography provide the same decision support to all users. The aim of this study was to investigate the potential of a context-sensitive CADe system which provides decision support guided by each user's focus of attention during visual search and reporting patterns for a specific case.

METHODS

An observer study for the detection of malignant masses in screening mammograms was conducted in which six radiologists evaluated 20 mammograms while wearing an eye-tracking device. Eye-position data and diagnostic decisions were collected for each radiologist and case they reviewed. These cases were subsequently analyzed with an in-house knowledge-based CADe system using two different modes: Conventional mode with a globally fixed decision threshold and context-sensitive mode with a location-variable decision threshold based on the radiologists' eye dwelling data and reporting information.

RESULTS

The CADe system operating in conventional mode had 85.7% per-image malignant mass sensitivity at 3.15 false positives per image (FPsI). The same system operating in context-sensitive mode provided personalized decision support at 85.7%-100% sensitivity and 0.35-0.40 FPsI to all six radiologists. Furthermore, context-sensitive CADe system could improve the radiologists' sensitivity and reduce their performance gap more effectively than conventional CADe.

CONCLUSIONS

Context-sensitive CADe support shows promise in delineating and reducing the radiologists' perceptual and cognitive errors in the diagnostic interpretation of screening mammograms more effectively than conventional CADe.