Performance of breast ultrasound computer-aided diagnosis: dependence on image selection

Deep learning-based classification of breast lesions using dynamic ultrasound video

PURPOSE

We intended to develop a deep-learning-based classification model based on breast ultrasound dynamic video, then evaluate its diagnostic performance in comparison with the classic model based on ultrasound static image and that of different radiologists.

METHOD

We collected 1000 breast lesions from 888 patients from May 2020 to December 2021. Each lesion contained two static images and two dynamic videos. We divided these lesions randomly into training, validation, and test sets by the ratio of 7:2:1. Two deep learning (DL) models, namely DL-video and DL-image, were developed based on 3D Resnet-50 and 2D Resnet-50 using 2000 dynamic videos and 2000 static images, respectively. Lesions in the test set were evaluated to compare the diagnostic performance of two models and six radiologists with different seniority.

RESULTS

The area under the curve of the DL-video model was significantly higher than those of the DL-image model (0.969 vs. 0.925, P = 0.0172) and six radiologists (0.969 vs. 0.779-0.912, P < 0.05). All radiologists performed better when evaluating the dynamic videos compared to the static images. Furthermore, radiologists performed better with increased seniority both in reading images and videos.

CONCLUSIONS

The DL-video model can discern more detailed spatial and temporal information for accurate classification of breast lesions than the conventional DL-image model and radiologists, and its clinical application can further improve the diagnosis of breast cancer.

Tumor Microenvironment and the Role of Artificial Intelligence in Breast Cancer Detection and Prognosis

A critical knowledge gap has been noted in breast cancer detection, prognosis, and evaluation between tumor microenvironment and associated neoplasm. Artificial intelligence (AI) has multiple subsets or methods for data extraction and evaluation, including artificial neural networking, which allows computational foundations, similar to neurons, to make connections and new neural pathways during data set training. Deep machine learning and AI hold great potential to accurately assess tumor microenvironment models employing vast data management techniques. Despite the significant potential AI holds, there is still much debate surrounding the appropriate and ethical curation of medical data from picture archiving and communication systems. AI output's clinical significance depends on its human predecessor's data training sets. Integration between biomarkers, risk factors, and imaging data will allow the best predictor models for patient-based outcomes.

Deep learning in medical imaging and radiation therapy

The goals of this review paper on deep learning (DL) in medical imaging and radiation therapy are to (a) summarize what has been achieved to date; (b) identify common and unique challenges, and strategies that researchers have taken to address these challenges; and (c) identify some of the promising avenues for the future both in terms of applications as well as technical innovations. We introduce the general principles of DL and convolutional neural networks, survey five major areas of application of DL in medical imaging and radiation therapy, identify common themes, discuss methods for dataset expansion, and conclude by summarizing lessons learned, remaining challenges, and future directions.

Fuzzy c-means segmentation of major vessels in angiographic images of stroke

Patients suffering from ischemic stroke develop varying degrees of pial arterial supply (PAS), which can affect patient response to reperfusion therapy and risk of hemorrhage. Since vessel segmentation may be an important part in identifying PAS, we present a fuzzy c-means (FCM) clustering method to segment major vessels in x-ray angiograms. Our approach consists of semiautomatic region of interest (ROI) delineation, separation of major vessels from capillary blush and/or background noise through FCM clustering, and identification of the major vessel category. This method was applied to a database of x-ray angiograms of 24 patients acquired at various frame rates. The ground truth for performance evaluation was the designation by an expert radiologist selecting image pixels as being vessel or nonvessel. From receiver operating characteristic (ROC) analysis, area under the ROC curve (AUC) was the performance metric in the task of distinguishing between major vessels and blush or background. When clustering data into three categories and performing FCM segmentation on each ROI separately, the AUC was 0.89 for the entire database and [Formula: see text] for all examined frame-rates. In conclusion, our method showed promising performance in identifying major vessels and is anticipated to become an integral part of automatic quantification of PAS.

Breast tumor classification in ultrasound images using neural networks with improved generalization methods

Mammography, scintimammography and ultrasound images have been used to increase the specificity of breast cancer image diagnosis. Concerning breast cancer image diagnosis with ultrasound, some results found in the literature show better performance of morphological features in breast cancer lesion differentiation and that a reduced set of features shows a better performance than a large set of features. In this study we evaluated the performance of neural network classifiers, with different training stop criteria: mean square error, early stop and regularization. The last two criteria were developed to improve neural network generalization. Different sets of morphological features were used as neural network inputs. Training sets comprised of 22, 8, 7, 6, 5 and 4 features were employed. To select reduced sets of features, a scalar selection technique with correlation was used. The best results obtained for accuracy and area under the ROC curve were 96.98% and 0.98, respectively. The performance obtained with all 22 features is slightly better than the one obtained with a reduced set of features.

BI-RADS Density Classification From Areometric and Volumetric Automatic Breast Density Measurements

RATIONALE AND OBJECTIVES

The aim of our study was to classify breast density using areometric and volumetric automatic measurements to best match Breast Imaging-Reporting and Data System (BI-RADS) density scores, and determine which technique best agrees with BI-RADS. Second, this study aimed to provide a set of threshold values for areometric and volumetric density to estimate BI-RADS categories.

MATERIALS AND METHODS

We randomly selected 537 full-field digital mammography examinations from a population-based screening program. Five radiologists assessed breast density using BI-RADS with all views available. A commercial program calculated areometric and volumetric breast density automatically. We compared automatically calculated density to all BI-RADS density thresholds using area under the receiver operating characteristic curve, and used Youden's index to estimate thresholds in automatic densities, with matching sensitivity and specificity. The 95% confidence intervals were estimated by bootstrapping.

RESULTS

Areometric density correlated well with volumetric density (r(2) = 0.76, excluding outliers, n = 2). For the BI-RADS threshold between II and III, areometric and volumetric assessment showed about equal area under the curve (0.94 vs. 0.93). For the threshold between I and II, areometric assessment was better than volumetric assessment (0.91 vs. 0.86). For the threshold between III and IV, volumetric assessment was better than areometric assessment (0.97 vs. 0.92).

CONCLUSIONS

Volumetric assessment is equal to or better than areometric assessment for the most clinically relevant thresholds (ie, between scattered fibroglandular and heterogeneously dense, and between heterogeneously dense and extremely dense breasts). Thresholds found in this study can be applied in daily practice to automatic measurements of density to estimate BI-RADS classification.

Mammographic quantitative image analysis and biologic image composition for breast lesion characterization and classification

PURPOSE

To investigate whether biologic image composition of mammographic lesions can improve upon existing mammographic quantitative image analysis (QIA) in estimating the probability of malignancy.

METHODS

The study population consisted of 45 breast lesions imaged with dual-energy mammography prior to breast biopsy with final diagnosis resulting in 10 invasive ductal carcinomas, 5 ductal carcinomain situ, 11 fibroadenomas, and 19 other benign diagnoses. Analysis was threefold: (1) The raw low-energy mammographic images were analyzed with an established in-house QIA method, "QIA alone," (2) the three-compartment breast (3CB) composition measure-derived from the dual-energy mammography-of water, lipid, and protein thickness were assessed, "3CB alone", and (3) information from QIA and 3CB was combined, "QIA + 3CB." Analysis was initiated from radiologist-indicated lesion centers and was otherwise fully automated. Steps of the QIA and 3CB methods were lesion segmentation, characterization, and subsequent classification for malignancy in leave-one-case-out cross-validation. Performance assessment included box plots, Bland-Altman plots, and Receiver Operating Characteristic (ROC) analysis.

RESULTS

The area under the ROC curve (AUC) for distinguishing between benign and malignant lesions (invasive and DCIS) was 0.81 (standard error 0.07) for the "QIA alone" method, 0.72 (0.07) for "3CB alone" method, and 0.86 (0.04) for "QIA+3CB" combined. The difference in AUC was 0.043 between "QIA + 3CB" and "QIA alone" but failed to reach statistical significance (95% confidence interval [-0.17 to + 0.26]).

CONCLUSIONS

In this pilot study analyzing the new 3CB imaging modality, knowledge of the composition of breast lesions and their periphery appeared additive in combination with existing mammographic QIA methods for the distinction between different benign and malignant lesion types.

Computerized detection of breast cancer on automated breast ultrasound imaging of women with dense breasts

PURPOSE

Develop a computer-aided detection method and investigate its feasibility for detection of breast cancer in automated 3D ultrasound images of women with dense breasts.

METHODS

The HIPAA compliant study involved a dataset of volumetric ultrasound image data, "views," acquired with an automated U-Systems Somo●V(®) ABUS system for 185 asymptomatic women with dense breasts (BI-RADS Composition/Density 3 or 4). For each patient, three whole-breast views (3D image volumes) per breast were acquired. A total of 52 patients had breast cancer (61 cancers), diagnosed through any follow-up at most 365 days after the original screening mammogram. Thirty-one of these patients (32 cancers) had a screening-mammogram with a clinically assigned BI-RADS Assessment Category 1 or 2, i.e., were mammographically negative. All software used for analysis was developed in-house and involved 3 steps: (1) detection of initial tumor candidates, (2) characterization of candidates, and (3) elimination of false-positive candidates. Performance was assessed by calculating the cancer detection sensitivity as a function of the number of "marks" (detections) per view.

RESULTS

At a single mark per view, i.e., six marks per patient, the median detection sensitivity by cancer was 50.0% (16/32) ± 6% for patients with a screening mammogram-assigned BI-RADS category 1 or 2--similar to radiologists' performance sensitivity (49.9%) for this dataset from a prior reader study--and 45.9% (28/61) ± 4% for all patients.

CONCLUSIONS

Promising detection sensitivity was obtained for the computer on a 3D ultrasound dataset of women with dense breasts at a rate of false-positive detections that may be acceptable for clinical implementation.

Interreader scoring variability in an observer study using dual-modality imaging for breast cancer detection in women with dense breasts

RATIONALE AND OBJECTIVES

To evaluate variability in the clinical assessment of breast images, we evaluated scoring behavior of radiologists in a retrospective reader study combining x-ray mammography (XRM) and three-dimensional automated breast ultrasound (ABUS) for breast cancer detection in women with dense breasts.

METHODS

The study involved 17 breast radiologists in a sequential study design with readers first interpreting XRM-alone followed by an interpretation of combined XRM + ABUS. Each interpretation included a forced Breast Imaging Reporting and Data System scale and a likelihood that the woman had breast cancer. The analysis included 164 asymptomatic patients, including 31 breast cancer patients, with dense breasts and a negative screening XRM. Of interest were interreader scoring variability for XRM-alone, XRM + ABUS, and the sequential effect. In addition, a simulated double reading by pairs of readers of XRM + ABUS was investigated. Performance analysis included receiver operating characteristic analysis, percentile analysis, and κ statistics. Bootstrapping was used to determine statistical significance.

RESULTS

The median change in area under the receiver operating characteristic curve after ABUS interpretation was 0.12 (range 0.04-0.19). Reader agreement was fair with the median interreader κ being 0.26 (0.05-0.48) for XRM-alone and 0.34 (0.11-0.55) for XRM + ABUS (95% confidence interval for the difference in κ, 0.06-0.11). Simulated double reading of XRM + ABUS demonstrated tradeoffs in sensitivity and specificity, but conservative simulated double reading resulted in a significant improvement in both sensitivity (16.7%) and specificity (7.6%) with respect to XRM-alone.

CONCLUSION

A modest, but statistically significant, increase in interreader agreement was observed after interpretation of ABUS.

MRI pyschophysics: an experimental framework relating image quality to diagnostic performance metrics

PURPOSE

To determine the minimal image quality needed to preserve diagnostic performance relative to arthroscopy in the knee.

MATERIALS AND METHODS

Synthetic noise was added to images from clinical MRI scans (three-dimensional SPACE pulse sequence; Siemens) from five patients who had undergone knee MRI with arthroscopic follow-up, resulting in 25 simulated sets of images with standardized signal-to-noise ratios (SNRs) of 1, 2, 5, 10, or 20. All cases were scored by four musculoskeletal radiologists progressing from low to high SNR and grading all cartilage surfaces, major ligaments and menisci on a 5-point scale. Receiver operator characteristic (ROC) curves were constructed for the detection of meniscal tears and cartilage abnormalities. The area under the ROC curve (AUC) was determined for each structure at each SNR level. In addition, reader confidence was measured and pairwise comparisons across SNR levels were performed. Results were compared with arthroscopy as the reference standard.

RESULTS

ROC AUC was maximized for meniscal tears at SNR = 5 (structure specific CNR = 3.2) and for cartilage abnormalities at SNR = 10 (CNR = 4.2). Observer confidence was maximized for menisci at SNR = 5 (CNR = 8.0), for ligaments at SNR = 10 (CNR = 13.6) and cartilage at SNR = 10 (CNR = 8.2).

CONCLUSION

For 3D isotropic imaging in the knee, images with SNR < 10 or CNR < 10 should be rejected as nondiagnostic.

Evaluation of the accuracy of a computer-aided diagnosis (CAD) system in breast ultrasound according to the radiologist's experience

RATIONALE AND OBJECTIVES

The aim of this study was to evaluate the performance of a computer-aided diagnosis (CAD) system for breast ultrasound to improve the characterization of breast lesions detected on ultrasound by junior and senior radiologists.

MATERIALS AND METHODS

One hundred sixty ultrasound breast lesions were randomly reviewed blindly by four radiologists with different levels of expertise (from 20 years [radiologist A] to 4 months [radiologist D]), with and without the help of an ultrasound CAD system (B-CAD version 2). All lesions had been biopsied. Sensitivity and specificity with and without CAD were calculated for each radiologist for the following evaluation criteria: Breast Imaging Reporting and Data System category and the final diagnosis (benign or malignant). Intrinsic sensitivity, specificity, and predictive values of CAD alone were also calculated.

RESULTS

CAD detected all cancers, and its use increased radiologists' sensitivity scores when this was possible (with vs without CAD: radiologist A, 99% vs 99%; radiologist B, 96% vs 87%; radiologist C, 95% vs 88%; radiologist D, 91% vs 88%). Seven additional cancers were diagnosed. However, the low specificity of CAD (48%) decreased the specificity of radiologists, especially of the more experienced among them (with vs without CAD: radiologist A, 46% vs 70%; radiologist B, 58% vs 80%; radiologist C, 57% vs 69%; radiologist D, 71% vs 71%).

CONCLUSIONS

CAD for breast ultrasound appears to be a useful tool for improving the diagnosis of malignant lesions for junior radiologists. Nevertheless, its low specificity must be taken into account to limit biopsies of benign lesions.

Repeatability in computer-aided diagnosis: application to breast cancer diagnosis on sonography

PURPOSE

The aim of this study was to investigate the concept of repeatability in a case-based performance evaluation of two classifiers commonly used in computer-aided diagnosis in the task of distinguishing benign from malignant lesions.

METHODS

The authors performed .632+ bootstrap analyses using a data set of 1251 sonographic lesions of which 212 were malignant. Several analyses were performed investigating the impact of sample size and number of bootstrap iterations. The classifiers investigated were a Bayesian neural net (BNN) with five hidden units and linear discriminant analysis (LDA). Both used the same four input lesion features. While the authors did evaluate classifier performance using receiver operating characteristic (ROC) analysis, the main focus was to investigate case-based performance based on the classifier output for individual cases, i.e., the classifier outputs for each test case measured over the bootstrap iterations. In this case-based analysis, the authors examined the classifier output variability and linked it to the concept of repeatability. Repeatability was assessed on the level of individual cases, overall for all cases in the data set, and regarding its dependence on the case-based classifier output. The impact of repeatability was studied when aiming to operate at a constant sensitivity or specificity and when aiming to operate at a constant threshold value for the classifier output.

RESULTS

The BNN slightly outperformed the LDA with an area under the ROC curve of 0.88 versus 0.85 (p < 0.05). In the repeatability analysis on an individual case basis, it was evident that different cases posed different degrees of difficulty to each classifier as measured by the by-case output variability. When considering the entire data set, however, the overall repeatability of the BNN classifier was lower than for the LDA classifier, i.e., the by-case variability for the BNN was higher. The dependence of the by-case variability on the average by-case classifier output was markedly different for the classifiers. The BNN achieved the lowest variability (best repeatability) when operating at high sensitivity (> 90%) and low specificity (< 66%), while the LDA achieved this at moderate sensitivity (approximately 74%) and specificity (approximately 84%). When operating at constant 90% sensitivity or constant 90% specificity, the width of the 95% confidence intervals for the corresponding classifier output was considerable for both classifiers and increased for smaller sample sizes. When operating at a constant threshold value for the classifier output, the width of the 95% confidence intervals for the corresponding sensitivity and specificity ranged from 9 percentage points (pp) to 30 pp.

CONCLUSIONS

The repeatability of the classifier output can have a substantial effect on the obtained sensitivity and specificity. Knowledge of classifier repeatability, in addition to overall performance level, is important for successful translation and implementation of computer-aided diagnosis in clinical decision making.

Computer-aided diagnostic models in breast cancer screening

Mammography is the most common modality for breast cancer detection and diagnosis and is often complemented by ultrasound and MRI. However, similarities between early signs of breast cancer and normal structures in these images make detection and diagnosis of breast cancer a difficult task. To aid physicians in detection and diagnosis, computer-aided detection and computer-aided diagnostic (CADx) models have been proposed. A large number of studies have been published for both computer-aided detection and CADx models in the last 20 years. The purpose of this article is to provide a comprehensive survey of the CADx models that have been proposed to aid in mammography, ultrasound and MRI interpretation. We summarize the noteworthy studies according to the screening modality they consider and describe the type of computer model, input data size, feature selection method, input feature type, reference standard and performance measures for each study. We also list the limitations of the existing CADx models and provide several possible future research directions.

Breast US computer-aided diagnosis system: robustness across urban populations in South Korea and the United States

PURPOSE

To evaluate the robustness of a breast ultrasonographic (US) computer-aided diagnosis (CAD) system in terms of its performance across different patient populations.

MATERIALS AND METHODS

Three US databases were analyzed for this study: one South Korean and two United States databases. All three databases were utilized in an institutional review board-approved and HIPAA-compliant manner. Round-robin analysis and independent testing were performed to evaluate the performance of a computerized breast cancer classification scheme across the databases. Receiver operating characteristic (ROC) analysis was used to evaluate performance differences.

RESULTS

The round-robin analyses of each database demonstrated similar results, with areas under the ROC curve ranging from 0.88 (95% confidence interval [CI]: 0.820, 0.918) to 0.91 (95% CI: 0.86, 0.95). The independent testing of each database, however, indicated that although the performances were similar, the range in areas under the ROC curve (from 0.79 [95% CI: 0.730, 0.842] to 0.87 [95% CI: 0.794, 0.923]) was wider than that with the round-robin tests. However, the only instances in which statistically significant differences in performance were demonstrated occurred when the Korean database was used in a testing capacity in independent testing.

CONCLUSION

The few observed statistically significant differences in performance indicated that while the US features used by the system were useful across the databases, their relative importance differed. In practice, this means that a CAD system may need to be adjusted when applied to a different population.