X-ray source motion blur modeling and deblurring with generative diffusion for digital breast tomosynthesis

Objective. Digital breast tomosynthesis (DBT) has significantly improved the diagnosis of breast cancer due to its high sensitivity and specificity in detecting breast lesions compared to two-dimensional mammography. However, one of the primary challenges in DBT is the image blur resulting from x-ray source motion, particularly in DBT systems with a source in continuous-motion mode. This motion-induced blur can degrade the spatial resolution of DBT images, potentially affecting the visibility of subtle lesions such as microcalcifications.Approach. We addressed this issue by deriving an analytical in-plane source blur kernel for DBT images based on imaging geometry and proposing a post-processing image deblurring method with a generative diffusion model as an image prior.Main results. We showed that the source blur could be approximated by a shift-invariant kernel over the DBT slice at a given height above the detector, and we validated the accuracy of our blur kernel modeling through simulation. We also demonstrated the ability of the diffusion model to generate realistic DBT images. The proposed deblurring method successfully enhanced spatial resolution when applied to DBT images reconstructed with detector blur and correlated noise modeling.Significance. Our study demonstrated the advantages of modeling the imaging system components such as source motion blur for improving DBT image quality.

Detection of Severe Lung Infection on Chest Radiographs of COVID-19 Patients: Robustness of AI Models across Multi-Institutional Data

The diagnosis of severe COVID-19 lung infection is important because it carries a higher risk for the patient and requires prompt treatment with oxygen therapy and hospitalization while those with less severe lung infection often stay on observation. Also, severe infections are more likely to have long-standing residual changes in their lungs and may need follow-up imaging. We have developed deep learning neural network models for classifying severe vs. non-severe lung infections in COVID-19 patients on chest radiographs (CXR). A deep learning U-Net model was developed to segment the lungs. Inception-v1 and Inception-v4 models were trained for the classification of severe vs. non-severe COVID-19 infection. Four CXR datasets from multi-country and multi-institutional sources were used to develop and evaluate the models. The combined dataset consisted of 5748 cases and 6193 CXR images with physicians' severity ratings as reference standard. The area under the receiver operating characteristic curve (AUC) was used to evaluate model performance. We studied the reproducibility of classification performance using the different combinations of training and validation data sets. We also evaluated the generalizability of the trained deep learning models using both independent internal and external test sets. The Inception-v1 based models achieved AUC ranging between 0.81 ± 0.02 and 0.84 ± 0.0, while the Inception-v4 models achieved AUC in the range of 0.85 ± 0.06 and 0.89 ± 0.01, on the independent test sets, respectively. These results demonstrate the promise of using deep learning models in differentiating COVID-19 patients with severe from non-severe lung infection on chest radiographs.

Artificial intelligence in medicine: mitigating risks and maximizing benefits via quality assurance, quality control, and acceptance testing

The adoption of artificial intelligence (AI) tools in medicine poses challenges to existing clinical workflows. This commentary discusses the necessity of context-specific quality assurance (QA), emphasizing the need for robust QA measures with quality control (QC) procedures that encompass (1) acceptance testing (AT) before clinical use, (2) continuous QC monitoring, and (3) adequate user training. The discussion also covers essential components of AT and QA, illustrated with real-world examples. We also highlight what we see as the shared responsibility of manufacturers or vendors, regulators, healthcare systems, medical physicists, and clinicians to enact appropriate testing and oversight to ensure a safe and equitable transformation of medicine through AI.

Model-based deep CNN-regularized reconstruction for digital breast tomosynthesis with a task-based CNN image assessment approach

Objective. Digital breast tomosynthesis (DBT) is a quasi-three-dimensional breast imaging modality that improves breast cancer screening and diagnosis because it reduces fibroglandular tissue overlap compared with 2D mammography. However, DBT suffers from noise and blur problems that can lower the detectability of subtle signs of cancers such as microcalcifications (MCs). Our goal is to improve the image quality of DBT in terms of image noise and MC conspicuity.Approach. We proposed a model-based deep convolutional neural network (deep CNN or DCNN) regularized reconstruction (MDR) for DBT. It combined a model-based iterative reconstruction (MBIR) method that models the detector blur and correlated noise of the DBT system and the learning-based DCNN denoiser using the regularization-by-denoising framework. To facilitate the task-based image quality assessment, we also proposed two DCNN tools for image evaluation: a noise estimator (CNN-NE) trained to estimate the root-mean-square (RMS) noise of the images, and an MC classifier (CNN-MC) as a DCNN model observer to evaluate the detectability of clustered MCs in human subject DBTs.Main results. We demonstrated the efficacies of CNN-NE and CNN-MC on a set of physical phantom DBTs. The MDR method achieved low RMS noise and the highest detection area under the receiver operating characteristic curve (AUC) rankings evaluated by CNN-NE and CNN-MC among the reconstruction methods studied on an independent test set of human subject DBTs.Significance. The CNN-NE and CNN-MC may serve as a cost-effective surrogate for human observers to provide task-specific metrics for image quality comparisons. The proposed reconstruction method shows the promise of combining physics-based MBIR and learning-based DCNNs for DBT image reconstruction, which may potentially lead to lower dose and higher sensitivity and specificity for MC detection in breast cancer screening and diagnosis.

Deep learning denoising of digital breast tomosynthesis: Observer performance study of the effect on detection of microcalcifications in breast phantom images

BACKGROUND

The noise in digital breast tomosynthesis (DBT) includes x-ray quantum noise and detector readout noise. The total radiation dose of a DBT scan is kept at about the level of a digital mammogram but the detector noise is increased due to acquisition of multiple projections. The high noise can degrade the detectability of subtle lesions, specifically microcalcifications (MCs).

PURPOSE

We previously developed a deep-learning-based denoiser to improve the image quality of DBT. In the current study, we conducted an observer performance study with breast radiologists to investigate the feasibility of using deep-learning-based denoising to improve the detection of MCs in DBT.

METHODS

We have a modular breast phantom set containing seven 1-cm-thick heterogeneous 50% adipose/50% fibroglandular slabs custom-made by CIRS, Inc. (Norfolk, VA). We made six 5-cm-thick breast phantoms embedded with 144 simulated MC clusters of four nominal speck sizes (0.125-0.150, 0.150-0.180, 0.180-0.212, 0.212-0.250 mm) at random locations. The phantoms were imaged with a GE Pristina DBT system using the automatic standard (STD) mode. The phantoms were also imaged with the STD+ mode that increased the average glandular dose by 54% to be used as a reference condition for comparison of radiologists' reading. Our previously trained and validated denoiser was deployed to the STD images to obtain a denoised DBT set (dnSTD). Seven breast radiologists participated as readers to detect the MCs in the DBT volumes of the six phantoms under the three conditions (STD, STD+, dnSTD), totaling 18 DBT volumes. Each radiologist read all the 18 DBT volumes sequentially, which were arranged in a different order for each reader in a counter-balanced manner to minimize any potential reading order effects. They marked the location of each detected MC cluster and provided a conspicuity rating and their confidence level for the perceived cluster. The visual grading characteristics (VGC) analysis was used to compare the conspicuity ratings and the confidence levels of the radiologists for the detection of MCs.

RESULTS

The average sensitivities over all MC speck sizes were 65.3%, 73.2%, and 72.3%, respectively, for the radiologists reading the STD, dnSTD, and STD+ volumes. The sensitivity for dnSTD was significantly higher than that for STD (p < 0.005, two-tailed Wilcoxon signed rank test) and comparable to that for STD+. The average false positive rates were 3.9 ± 4.6, 2.8 ± 3.7, and 2.7 ± 3.9 marks per DBT volume, respectively, for reading the STD, dnSTD, and STD+ images but the difference between dnSTD and STD or STD+ did not reach statistical significance. The overall conspicuity ratings and confidence levels by VGC analysis for dnSTD were significantly higher than those for both STD and STD+ (p ≤ 0.001). The critical alpha value for significance was adjusted to be 0.025 with Bonferroni correction.

CONCLUSIONS

This observer study using breast phantom images showed that deep-learning-based denoising has the potential to improve the detection of MCs in noisy DBT images and increase radiologists' confidence in differentiating noise from MCs without increasing radiation dose. Further studies are needed to evaluate the generalizability of these results to the wide range of DBTs from human subjects and patient populations in clinical settings.

Survival Prediction of Patients with Bladder Cancer after Cystectomy Based on Clinical, Radiomics, and Deep-Learning Descriptors

Accurate survival prediction for bladder cancer patients who have undergone radical cystectomy can improve their treatment management. However, the existing predictive models do not take advantage of both clinical and radiological imaging data. This study aimed to fill this gap by developing an approach that leverages the strengths of clinical (C), radiomics (R), and deep-learning (D) descriptors to improve survival prediction. The dataset comprised 163 patients, including clinical, histopathological information, and CT urography scans. The data were divided by patient into training, validation, and test sets. We analyzed the clinical data by a nomogram and the image data by radiomics and deep-learning models. The descriptors were input into a BPNN model for survival prediction. The AUCs on the test set were (C): 0.82 ± 0.06, (R): 0.73 ± 0.07, (D): 0.71 ± 0.07, (CR): 0.86 ± 0.05, (CD): 0.86 ± 0.05, and (CRD): 0.87 ± 0.05. The predictions based on D and CRD descriptors showed a significant difference (p = 0.007). For Kaplan-Meier survival analysis, the deceased and alive groups were stratified successfully by C (p < 0.001) and CRD (p < 0.001), with CRD predicting the alive group more accurately. The results highlight the potential of combining C, R, and D descriptors to accurately predict the survival of bladder cancer patients after cystectomy.

Improved Repeatability of Mouse Tibia Volume Segmentation in Murine Myelofibrosis Model Using Deep Learning

A murine model of myelofibrosis in tibia was used in a co-clinical trial to evaluate segmentation methods for application of image-based biomarkers to assess disease status. The dataset (32 mice with 157 3D MRI scans including 49 test-retest pairs scanned on consecutive days) was split into approximately 70% training, 10% validation, and 20% test subsets. Two expert annotators (EA1 and EA2) performed manual segmentations of the mouse tibia (EA1: all data; EA2: test and validation). Attention U-net (A-U-net) model performance was assessed for accuracy with respect to EA1 reference using the average Jaccard index (AJI), volume intersection ratio (AVI), volume error (AVE), and Hausdorff distance (AHD) for four training scenarios: full training, two half-splits, and a single-mouse subsets. The repeatability of computer versus expert segmentations for tibia volume of test-retest pairs was assessed by within-subject coefficient of variance (%wCV). A-U-net models trained on full and half-split training sets achieved similar average accuracy (with respect to EA1 annotations) for test set: AJI = 83-84%, AVI = 89-90%, AVE = 2-3%, and AHD = 0.5 mm-0.7 mm, exceeding EA2 accuracy: AJ = 81%, AVI = 83%, AVE = 14%, and AHD = 0.3 mm. The A-U-net model repeatability wCV [95% CI]: 3 [2, 5]% was notably better than that of expert annotators EA1: 5 [4, 9]% and EA2: 8 [6, 13]%. The developed deep learning model effectively automates murine bone marrow segmentation with accuracy comparable to human annotators and substantially improved repeatability.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.

Recursive Training Strategy for a Deep Learning Network for Segmentation of Pathology Nuclei With Incomplete Annotation

This study developed a recursive training strategy to train a deep learning model for nuclei detection and segmentation using incomplete annotation. A dataset of 141 H&E stained breast cancer pathologic images with incomplete annotation was randomly split into training/validation set and test set of 89 and 52 images, respectively. The positive training samples were extracted at each annotated cell and augmented with affine translation. The negative training samples were selected from the non-cellular regions free of nuclei using a histogram-based semi-automatic method. A U-Net model was initially trained by minimizing a custom loss function. After the first stage of training, the trained U-Net model was applied to the images in the training set in an inference mode. The U-Net segmented objects with high quality were selected by a semi-automated method. Combining the newly selected high quality objects with the annotated nuclei and the previously generated negative samples, the U-Net model was retrained recursively until the stopping criteria were satisfied. For the 52 test images, the U-Net trained with and without using our recursive training method achieved a sensitivity of 90.3% and 85.3% for nuclei detection, respectively. For nuclei segmentation, the average Dice coefficient and average Jaccard index were 0.831±0.213 and 0.750±0.217, 0.780±0.270 and 0.697±0.264, for U-Net with and without recursive training, respectively. The improvement achieved by our proposed method was statistically significant (P < 0.05). In conclusion, our recursive training method effectively enlarged the set of annotated objects for training the deep learning model and further improved the detection and segmentation performance.

Computerized Decision Support for Bladder Cancer Treatment Response Assessment in CT Urography: Effect on Diagnostic Accuracy in Multi-Institution Multi-Specialty Study

This observer study investigates the effect of computerized artificial intelligence (AI)-based decision support system (CDSS-T) on physicians’ diagnostic accuracy in assessing bladder cancer treatment response. The performance of 17 observers was evaluated when assessing bladder cancer treatment response without and with CDSS-T using pre- and post-chemotherapy CTU scans in 123 patients having 157 pre- and post-treatment cancer pairs. The impact of cancer case difficulty, observers’ clinical experience, institution affiliation, specialty, and the assessment times on the observers’ diagnostic performance with and without using CDSS-T were analyzed. It was found that the average performance of the 17 observers was significantly improved (p = 0.002) when aided by the CDSS-T. The cancer case difficulty, institution affiliation, specialty, and the assessment times influenced the observers’ performance without CDSS-T. The AI-based decision support system has the potential to improve the diagnostic accuracy in assessing bladder cancer treatment response and result in more consistent performance among all physicians.

Effect of Dose Level on Radiologists' Detection of Microcalcifications in Digital Breast Tomosynthesis: An Observer Study with Breast Phantoms

OBJECTIVES

To compare radiologists' sensitivity, confidence level, and reading efficiency of detecting microcalcifications in digital breast tomosynthesis (DBT) at two clinically relevant dose levels.

MATERIALS AND METHODS

Six 5-cm-thick heterogeneous breast phantoms embedded with a total of 144 simulated microcalcification clusters of four speck sizes were imaged at two dose modes by a clinical DBT system. The DBT volumes at the two dose levels were read independently by six MQSA radiologists and one fellow with 1-33 years (median 12 years) of experience in a fully-crossed counter-balanced manner. The radiologist located each potential cluster and rated its conspicuity and his/her confidence that the marked location contained a cluster. The differences in the results between the two dose modes were analyzed by two-tailed paired t-test.

RESULTS

Compared to the lower-dose mode, the average glandular dose in the higher-dose mode for the 5-cm phantoms increased from 1.34 to 2.07 mGy. The detection sensitivity increased for all speck sizes and significantly for the two smaller sizes (p <0.05). An average of 13.8% fewer false positive clusters was marked. The average conspicuity rating and the radiologists' confidence level were higher for all speck sizes and reached significance (p <0.05) for the three larger sizes. The average reading time per detected cluster reduced significantly (p <0.05) by an average of 13.2%.

CONCLUSION

For a 5-cm-thick breast, an increase in average glandular dose from 1.34 to 2.07 mGy for DBT imaging increased the conspicuity of microcalcifications, improved the detection sensitivity by radiologists, increased their confidence levels, reduced false positive detections, and increased the reading efficiency.

AI in medical physics: guidelines for publication

The Abstract is intended to provide a concise summary of the study and its scientific findings. For AI/ML applications in medical physics, a problem statement and rationale for utilizing these algorithms are necessary while highlighting the novelty of the approach. A brief numerical description of how the data are partitioned into subsets for training of the AI/ML algorithm, validation (including tuning of parameters), and independent testing of algorithm performance is required. This is to be followed by a summary of the results and statistical metrics that quantify the performance of the AI/ML algorithm.

Deep Convolutional Neural Network With Adversarial Training for Denoising Digital Breast Tomosynthesis Images

Digital breast tomosynthesis (DBT) is a quasi-three-dimensional imaging modality that can reduce false negatives and false positives in mass lesion detection caused by overlapping breast tissue in conventional two-dimensional (2D) mammography. The patient dose of a DBT scan is similar to that of a single 2D mammogram, while acquisition of each projection view adds detector readout noise. The noise is propagated to the reconstructed DBT volume, possibly obscuring subtle signs of breast cancer such as microcalcifications (MCs). This study developed a deep convolutional neural network (DCNN) framework for denoising DBT images with a focus on improving the conspicuity of MCs as well as preserving the ill-defined margins of spiculated masses and normal tissue textures. We trained the DCNN using a weighted combination of mean squared error (MSE) loss and adversarial loss. We configured a dedicated x-ray imaging simulator in combination with digital breast phantoms to generate realistic in silico DBT data for training. We compared the DCNN training between using digital phantoms and using real physical phantoms. The proposed denoising method improved the contrast-to-noise ratio (CNR) and detectability index (d') of the simulated MCs in the validation phantom DBTs. These performance measures improved with increasing training target dose and training sample size. Promising denoising results were observed on the transferability of the digital-phantom-trained denoiser to DBT reconstructed with different techniques and on a small independent test set of human subject DBT images.

Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets

Radiomic features are being increasingly studied for clinical applications. We aimed to assess the agreement among radiomic features when computed by several groups by using different software packages under very tightly controlled conditions, which included standardized feature definitions and common image data sets. Ten sites (9 from the NCI's Quantitative Imaging Network] positron emission tomography–computed tomography working group plus one site from outside that group) participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape, and texture. The common image data sets were: three 3D digital reference objects (DROs) and 10 patient image scans from the Lung Image Database Consortium data set using a specific lesion in each scan. Each object (DRO or lesion) was accompanied by an already-defined volume of interest, from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The coefficient of variation (CV), expressed as a percentage, was calculated across software packages for each feature on each object. Thirteen sets of results were obtained for the DROs and patient data sets. Five of the 9 features showed excellent agreement with CV < 1%; 1 feature had moderate agreement (CV < 10%), and 3 features had larger variations (CV ≥ 10%) even after attempts at harmonization of feature calculations. This work highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.

Intraobserver Variability in Bladder Cancer Treatment Response Assessment With and Without Computerized Decision Support

We evaluated the intraobserver variability of physicians aided by a computerized decision-support system for treatment response assessment (CDSS-T) to identify patients who show complete response to neoadjuvant chemotherapy for bladder cancer, and the effects of the intraobserver variability on physicians' assessment accuracy. A CDSS-T tool was developed that uses a combination of deep learning neural network and radiomic features from computed tomography (CT) scans to detect bladder cancers that have fully responded to neoadjuvant treatment. Pre- and postchemotherapy CT scans of 157 bladder cancers from 123 patients were collected. In a multireader, multicase observer study, physician-observers estimated the likelihood of pathologic T0 disease by viewing paired pre/posttreatment CT scans placed side by side on an in-house-developed graphical user interface. Five abdominal radiologists, 4 diagnostic radiology residents, 2 oncologists, and 1 urologist participated as observers. They first provided an estimate without CDSS-T and then with CDSS-T. A subset of cases was evaluated twice to study the intraobserver variability and its effects on observer consistency. The mean areas under the curves for assessment of pathologic T0 disease were 0.85 for CDSS-T alone, 0.76 for physicians without CDSS-T and improved to 0.80 for physicians with CDSS-T (P = .001) in the original evaluation, and 0.78 for physicians without CDSS-T and improved to 0.81 for physicians with CDSS-T (P = .010) in the repeated evaluation. The intraobserver variability was significantly reduced with CDSS-T (P < .0001). The CDSS-T can significantly reduce physicians' variability and improve their accuracy for identifying complete response of muscle-invasive bladder cancer to neoadjuvant chemotherapy.

Risks of feature leakage and sample size dependencies in deep feature extraction for breast mass classification

PURPOSE

Transfer learning is commonly used in deep learning for medical imaging to alleviate the problem of limited available data. In this work, we studied the risk of feature leakage and its dependence on sample size when using pretrained deep convolutional neural network (DCNN) as feature extractor for classification breast masses in mammography.

METHODS

Feature leakage occurs when the training set is used for feature selection and classifier modeling while the cost function is guided by the validation performance or informed by the test performance. The high-dimensional feature space extracted from pretrained DCNN suffers from the curse of dimensionality; feature subsets that can provide excessively optimistic performance can be found for the validation set or test set if the latter is allowed for unlimited reuse during algorithm development. We designed a simulation study to examine feature leakage when using DCNN as feature extractor for mass classification in mammography. Four thousand five hundred and seventy-seven unique mass lesions were partitioned by patient into three sets: 3222 for training, 508 for validation, and 847 for independent testing. Three pretrained DCNNs, AlexNet, GoogLeNet, and VGG16, were first compared using a training set in fourfold cross validation and one was selected as the feature extractor. To assess generalization errors, the independent test set was sequestered as truly unseen cases. A training set of a range of sizes from 10% to 75% was simulated by random drawing from the available training set in addition to 100% of the training set. Three commonly used feature classifiers, the linear discriminant, the support vector machine, and the random forest were evaluated. A sequential feature selection method was used to find feature subsets that could achieve high classification performance in terms of the area under the receiver operating characteristic curve (AUC) in the validation set. The extent of feature leakage and the impact of training set size were analyzed by comparison to the performance in the unseen test set.

RESULTS

All three classifiers showed large generalization error between the validation set and the independent sequestered test set at all sample sizes. The generalization error decreased as the sample size increased. At 100% of the sample size, one classifier achieved an AUC as high as 0.91 on the validation set while the corresponding performance on the unseen test set only reached an AUC of 0.72.

CONCLUSIONS

Our results demonstrate that large generalization errors can occur in AI tools due to feature leakage. Without evaluation on unseen test cases, optimistically biased performance may be reported inadvertently, and can lead to unrealistic expectations and reduce confidence for clinical implementation.

Variabilities in Reference Standard by Radiologists and Performance Assessment in Detection of Pulmonary Embolism in CT Pulmonary Angiography

Annotating lesion locations by radiologists' manual marking is a key step to provide reference standard for the training and testing of a computer-aided detection system by supervised machine learning. Inter-reader variability is not uncommon in readings even by expert radiologists. This study evaluated the variability of the radiologist-identified pulmonary emboli (PEs) to demonstrate the importance of improving the reliability of the reference standard by a multi-step process for performance evaluation. In an initial reading of 40 CTPA PE cases, two experienced thoracic radiologists independently marked the PE locations. For markings from the two radiologists that did not agree, each radiologist re-read the cases independently to assess the discordant markings. Finally, for markings that still disagreed after the second reading, the two radiologists read together to reach a consensus. The variability of radiologists was evaluated by analyzing the agreement between two radiologists. For the 40 cases, 475 and 514 PEs were identified by radiologists R1 and R2 in the initial independent readings, respectively. For a total of 545 marks by the two radiologists, 81.5% (444/545) of the marks agreed but 101 marks in 36 cases differed. After consensus, 65 (64.4%) and 36 (35.6%) of the 101 marks were determined to be true PEs and false positives (FPs), respectively. Of these, 48 and 17 were false negatives (FNs) and 14 and 22 were FPs by R1 and R2, respectively. Our study demonstrated that there is substantial variability in reference standards provided by radiologists, which impacts the performance assessment of a lesion detection system. Combination of multiple radiologists' readings and consensus is needed to improve the reliability of a reference standard.

Computer-aided diagnosis in the era of deep learning

Computer-aided diagnosis (CAD) has been a major field of research for the past few decades. CAD uses machine learning methods to analyze imaging and/or nonimaging patient data and makes assessment of the patient's condition, which can then be used to assist clinicians in their decision-making process. The recent success of the deep learning technology in machine learning spurs new research and development efforts to improve CAD performance and to develop CAD for many other complex clinical tasks. In this paper, we discuss the potential and challenges in developing CAD tools using deep learning technology or artificial intelligence (AI) in general, the pitfalls and lessons learned from CAD in screening mammography and considerations needed for future implementation of CAD or AI in clinical use. It is hoped that the past experiences and the deep learning technology will lead to successful advancement and lasting growth in this new era of CAD, thereby enabling CAD to deliver intelligent aids to improve health care.

Prediction of Disease Free Survival in Laryngeal and Hypopharyngeal Cancers Using CT Perfusion and Radiomic Features: A Pilot Study

(1) Purpose: The objective was to evaluate CT perfusion and radiomic features for prediction of one year disease free survival in laryngeal and hypopharyngeal cancer. (2) Method and Materials: This retrospective study included pre and post therapy CT neck studies in 36 patients with laryngeal/hypopharyngeal cancer. Tumor contouring was performed semi-autonomously by the computer and manually by two radiologists. Twenty-six radiomic features including morphological and gray-level features were extracted by an internally developed and validated computer-aided image analysis system. The five perfusion features analyzed included permeability surface area product (PS), blood flow (flow), blood volume (BV), mean transit time (MTT), and time-to-maximum (Tmax). One year persistent/recurrent disease data were obtained following the final treatment of definitive chemoradiation or after total laryngectomy. We performed a two-loop leave-one-out feature selection and linear discriminant analysis classifier with generation of receiver operating characteristic (ROC) curves and confidence intervals (CI). (3) Results: 10 patients (28%) had recurrence/persistent disease at 1 year. For prediction, the change in blood flow demonstrated a training AUC of 0.68 (CI 0.47-0.85) and testing AUC of 0.66 (CI 0.47-0.85). The best features selected were a combination of perfusion and radiomic features including blood flow and computer-estimated percent volume changes-training AUC of 0.68 (CI 0.5-0.85) and testing AUC of 0.69 (CI 0.5-0.85). The laryngoscopic percent change in volume was a poor predictor with a testing AUC of 0.4 (CI 0.16-0.57). (4) Conclusions: A combination of CT perfusion and radiomic features are potential predictors of one-year disease free survival in laryngeal and hypopharyngeal cancer patients.

Assessment of task-based performance from five clinical DBT systems using an anthropomorphic breast phantom

PURPOSE

Digital breast tomosynthesis (DBT) is a limited-angle tomographic breast imaging modality that can be used for breast cancer screening in conjunction with full-field digital mammography (FFDM) or synthetic mammography (SM). Currently, there are five commercial DBT systems that have been approved by the U.S. FDA for breast cancer screening, all varying greatly in design and imaging protocol. Because the systems are different in technical specifications, there is a need for a quantitative approach for assessing them. In this study, the DBT systems are assessed using a novel methodology with an inkjet-printed anthropomorphic phantom and four alternative forced choice (4AFC) study scheme.

METHOD

A breast phantom was fabricated using inkjet printing and parchment paper. The phantom contained 5-mm spiculated masses fabricated with potassium iodide (KI)-doped ink and microcalcifications (MCs) made with calcium hydroxyapatite. Images of the phantom were acquired on all five systems with DBT, FFDM, and SM modalities where available using beam settings under automatic exposure control. A 4AFC study was conducted to assess reader performance with each signal under each modality. Statistical analysis was performed on the data to determine proportion correct (PC), standard deviations, and levels of significance.

RESULTS

For masses, overall detection was highest with DBT. The difference in PC was statistically significant between DBT and SM for most systems. A relationship was observed between increasing PC and greater gantry span. For MCs, performance was highest with DBT and FFDM compared to SM. The difference between PC of DBT and PC of SM was statistically significant for all manufacturers.

CONCLUSIONS

This methodology represents a novel approach for evaluating systems. This study is the first of its kind to use an inkjet-printed anthropomorphic phantom with realistic signals to assess performance of clinical DBT imaging systems.

Digital Breast Tomosynthesis Slab Thickness: Impact on Reader Performance and Interpretation Time

Background Digital breast tomosynthesis (DBT) helps reduce recall rates and improve cancer detection compared with two-dimensional (2D) mammography but has a longer interpretation time. Purpose To evaluate the effect of DBT slab thickness and overlap on reader performance and interpretation time in the absence of 1-mm slices. Materials and Methods In this retrospective HIPAA-compliant multireader study of DBT examinations performed between August 2013 and July 2017, four fellowship-trained breast imaging radiologists blinded to final histologic findings interpreted DBT examinations by using a standard protocol (10-mm slabs with 5-mm overlap, 1-mm slices, synthetic 2D mammogram) and an experimental protocol (6-mm slabs with 3-mm overlap, synthetic 2D mammogram) with a crossover design. Among the 122 DBT examinations, 74 mammographic findings had final histologic findings, including 31 masses (26 malignant), 20 groups of calcifications (12 malignant), 18 architectural distortions (15 malignant), and five asymmetries (two malignant). Durations of reader interpretations were recorded. Comparisons were made by using receiver operating characteristic curves for diagnostic performance and paired t tests for continuous variables. Results Among 122 women, mean age was 58.6 years ± 10.1 (standard deviation). For detection of malignancy, areas under the receiver operating characteristic curves were similar between protocols (range, 0.83-0.94 vs 0.84-0.92; P ≥ .63). Mean DBT interpretation time was shorter with the experimental protocol for three of four readers (reader 1, 5.6 minutes ± 1.7 vs 4.7 minutes ± 1.4 [P < .001]; reader 2, 2.8 minutes ± 1.1 vs 2.3 minutes ± 1.0 [P = .001]; reader 3, 3.6 minutes ± 1.4 vs 3.3 minutes ± 1.3 [P = .17]; reader 4, 4.3 minutes ± 1.0 vs 3.8 minutes ± 1.1 [P ≤ .001]), with 72% reduction in both mean number of images and mean file size (P < .001 for both). Conclusion A digital breast tomosynthesis reconstruction protocol that uses 6-mm slabs with 3-mm overlap, without 1-mm slices, had similar diagnostic performance compared with the standard protocol and led to a reduced interpretation time for three of four readers. © RSNA, 2020 See also the editorial by Chang in this issue.

Pathologic categorization of lung nodules: Radiomic descriptors of CT attenuation distribution patterns of solid and subsolid nodules in low-dose CT

PURPOSE

Develop a quantitative image analysis method to characterize the heterogeneous patterns of nodule components for the classification of pathological categories of nodules.

MATERIALS AND METHODS

With IRB approval and permission of the National Lung Screening Trial (NLST) project, 103 subjects with low dose CT (LDCT) were used in this study. We developed a radiomic quantitative CT attenuation distribution descriptor (qADD) to characterize the heterogeneous patterns of nodule components and a hybrid model (qADD+) that combined qADD with subject demographic data and radiologist-provided nodule descriptors to differentiate aggressive tumors from indolent tumors or benign nodules with pathological categorization as reference standard. The classification performances of qADD and qADD + were evaluated and compared to the Brock and the Mayo Clinic models by analysis of the area under the receiver operating characteristic curve (AUC).

RESULTS

The radiomic features were consistently selected into qADDs to differentiate pathological invasive nodules from (1) preinvasive nodules, (2) benign nodules, and (3) the group of preinvasive and benign nodules, achieving test AUCs of 0.847 ± 0.002, 0.842 ± 0.002 and 0.810 ± 0.001, respectively. The qADD + obtained test AUCs of 0.867 ± 0.002, 0.888 ± 0.001 and 0.852 ± 0.001, respectively, which were higher than both the Brock and the Mayo Clinic models.

CONCLUSION

The pathologic invasiveness of lung tumors could be categorized according to the CT attenuation distribution patterns of the nodule components manifested on LDCT images, and the majority of invasive lung cancers could be identified at baseline LDCT scans.

Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis

Deep convolutional neural network (DCNN), now popularly called artificial intelligence (AI), has shown the potential to improve over previous computer-assisted tools in medical imaging developed in the past decades. A DCNN has millions of free parameters that need to be trained, but the training sample set is limited in size for most medical imaging tasks so that transfer learning is typically used. Automatic data mining may be an efficient way to enlarge the collected data set but the data can be noisy such as incorrect labels or even a wrong type of image. In this work we studied the generalization error of DCNN with transfer learning in medical imaging for the task of classifying malignant and benign masses on mammograms. With a finite available data set, we simulated a training set containing corrupted data or noisy labels. The balance between learning and memorization of the DCNN was manipulated by varying the proportion of corrupted data in the training set. The generalization error of DCNN was analyzed by the area under the receiver operating characteristic curve for the training and test sets and the weight changes after transfer learning. The study demonstrates that the transfer learning strategy of DCNN for such tasks needs to be designed properly, taking into consideration the constraints of the available training set having limited size and quality for the classification task at hand, to minimize memorization and improve generalizability.

CAD and AI for breast cancer-recent development and challenges

Computer-aided diagnosis (CAD) has been a popular area of research and development in the past few decades. In CAD, machine learning methods and multidisciplinary knowledge and techniques are used to analyze the patient information and the results can be used to assist clinicians in their decision making process. CAD may analyze imaging information alone or in combination with other clinical data. It may provide the analyzed information directly to the clinician or correlate the analyzed results with the likelihood of certain diseases based on statistical modeling of the past cases in the population. CAD systems can be developed to provide decision support for many applications in the patient care processes, such as lesion detection, characterization, cancer staging, treatment planning and response assessment, recurrence and prognosis prediction. The new state-of-the-art machine learning technique, known as deep learning (DL), has revolutionized speech and text recognition as well as computer vision. The potential of major breakthrough by DL in medical image analysis and other CAD applications for patient care has brought about unprecedented excitement of applying CAD, or artificial intelligence (AI), to medicine in general and to radiology in particular. In this paper, we will provide an overview of the recent developments of CAD using DL in breast imaging and discuss some challenges and practical issues that may impact the advancement of artificial intelligence and its integration into clinical workflow.

Deep Learning in Medical Image Analysis

Deep learning is the state-of-the-art machine learning approach. The success of deep learning in many pattern recognition applications has brought excitement and high expectations that deep learning, or artificial intelligence (AI), can bring revolutionary changes in health care. Early studies of deep learning applied to lesion detection or classification have reported superior performance compared to those by conventional techniques or even better than radiologists in some tasks. The potential of applying deep-learning-based medical image analysis to computer-aided diagnosis (CAD), thus providing decision support to clinicians and improving the accuracy and efficiency of various diagnostic and treatment processes, has spurred new research and development efforts in CAD. Despite the optimism in this new era of machine learning, the development and implementation of CAD or AI tools in clinical practice face many challenges. In this chapter, we will discuss some of these issues and efforts needed to develop robust deep-learning-based CAD tools and integrate these tools into the clinical workflow, thereby advancing towards the goal of providing reliable intelligent aids for patient care.

Effect of source blur on digital breast tomosynthesis reconstruction

PURPOSE

Most digital breast tomosynthesis (DBT) reconstruction methods neglect the blurring of the projection views caused by the finite size or motion of the x-ray focal spot. This paper studies the effect of source blur on the spatial resolution of reconstructed DBT using analytical calculation and simulation, and compares the influence of source blur over a range of blurred source sizes.

METHODS

Mathematically derived formulas describe the point spread function (PSF) of source blur on the detector plane as a function of the spatial locations of the finite-sized source and the object. By using the available technical parameters of some clinical DBT systems, we estimated the effective source sizes over a range of exposure time and DBT scan geometries. We used the CatSim simulation tool (GE Global Research, NY) to generate digital phantoms containing line pairs and beads at different locations and imaged with sources of four different sizes covering the range of potential source blur. By analyzing the relative contrasts of the test objects in the reconstructed images, we studied the effect of the source blur on the spatial resolution of DBT. Furthermore, we simulated a detector that rotated in synchrony with the source about the rotation center and calculated the spatial distribution of the blurring distance in the imaged volume to estimate its influence on source blur.

RESULTS

Calculations demonstrate that the PSF is highly shift-variant, making it challenging to accurately implement during reconstruction. The results of the simulated phantoms demonstrated that a typical finite-sized focal spot (~0.3 mm) will not affect the reconstructed image resolution if the x-ray tube is stationary during data acquisition. If the x-ray tube moves during exposure, the extra blur due to the source motion may degrade image resolution, depending on the effective size of the source along the direction of the motion. A detector that rotates in synchrony with the source does not reduce the influence of source blur substantially.

CONCLUSIONS

This study demonstrates that the extra source blur due to the motion of the x-ray tube during image acquisition substantially degrades the reconstructed image resolution. This effect cannot be alleviated by rotating the detector in synchrony with the source. The simulation results suggest that there are potential benefits of modeling the source blur in image reconstruction for DBT systems using continuous-motion acquisition mode.

Diagnostic Accuracy of CT for Prediction of Bladder Cancer Treatment Response with and without Computerized Decision Support

RATIONALE AND OBJECTIVES

To evaluate whether a computed tomography (CT)-based computerized decision-support system for muscle-invasive bladder cancer treatment response assessment (CDSS-T) can improve identification of patients who have responded completely to neoadjuvant chemotherapy.

MATERIALS AND METHODS

Following Institutional Review Board approval, pre-chemotherapy and post-chemotherapy CT scans of 123 subjects with 157 muscle-invasive bladder cancer foci were collected retrospectively. CT data were analyzed with a CDSS-T that uses a combination of deep-learning convolutional neural network and radiomic features to distinguish muscle-invasive bladder cancers that have fully responded to neoadjuvant treatment from those that have not. Leave-one-case-out cross-validation was used to minimize overfitting. Five attending abdominal radiologists, four diagnostic radiology residents, two attending oncologists, and one attending urologist estimated the likelihood of pathologic T0 disease (complete response) by viewing paired pre/post-treatment CT scans placed side-by-side on an internally-developed graphical user interface. The observers provided an estimate without use of CDSS-T and then were permitted to revise their estimate after a CDSS-T-derived likelihood score was displayed. Observer estimates were analyzed with multi-reader, multi-case receiver operating characteristic methodology. The area under the curve (AUC) and the statistical significance of the difference were estimated.

RESULTS

The mean AUCs for assessment of pathologic T0 disease were 0.80 for CDSS-T alone, 0.74 for physicians not using CDSS-T, and 0.77 for physicians using CDSS-T. The increase in the physicians' performance was statistically significant (P < .05).

CONCLUSION

CDSS-T improves physician performance for identifying complete response of muscle-invasive bladder cancer to neoadjuvant chemotherapy.

Automated pectoral muscle identification on MLO-view mammograms: Comparison of deep neural network to conventional computer vision

OBJECTIVES

The aim of this study was to develop a fully automated deep learning approach for identification of the pectoral muscle on mediolateral oblique (MLO) view mammograms and evaluate its performance in comparison to our previously developed texture-field orientation (TFO) method using conventional image feature analysis. Pectoral muscle segmentation is an important step for automated image analyses such as breast density or parenchymal pattern classification, lesion detection, and multiview correlation.

MATERIALS AND METHODS

Institutional Review Board (IRB) approval was obtained before data collection. A dataset of 729 MLO-view mammograms including 637 digitized film mammograms (DFM) and 92 digital mammograms (DM) from our previous study was used for the training and validation of our deep convolutional neural network (DCNN) segmentation method. In addition, we collected an independent set of 203 DMs from 131 patients for testing. The film mammograms were digitized at a pixel size of 50 μm × 50 μm with a Lumiscan digitizer. All DMs were acquired with GE systems at a pixel size of 100 μm × 100 μm. An experienced MQSA radiologist manually drew the pectoral muscle boundary on each mammogram as the reference standard. We trained the DCNN to estimate a probability map of the pectoral muscle region on mammograms. The DCNN consisted of a contracting path to capture multiresolution image context and a symmetric expanding path for prediction of the pectoral muscle region. Three DCNN structures were compared for automated identification of pectoral muscles. Tenfold cross-validation was used in training of the DCNNs. After training, we applied the ten trained models during cross-validation to the independent DM test set. The predicted pectoral muscle region of each test DM was obtained as the mean probability map by averaging the ensemble of probability maps from the ten models. The DCNN-segmented pectoral muscle was evaluated by three performance measures relative to the reference standard: (a) the percent overlap area (POA) of the pectoral muscle regions, (b) the Hausdorff distance (Hdist), and (c) the average Euclidean distance (AvgDist) between the boundaries. The results were compared to those obtained with the TFO method, used as our baseline. A two-tailed paired t test was performed to examine the significance in the differences between the DCNN and the baseline.

RESULTS

In the ten test partitions of the cross-validation set, the DCNN achieved a mean POA of 96.5 ± 2.9%, a mean Hdist of 2.26 ± 1.31 mm, and a mean AvgDist of 0.78 ± 0.58 mm, while the corresponding measures by the baseline method were 94.2 ± 4.8%, 3.69 ± 2.48 mm, and 1.30 ± 1.22 mm, respectively. For the independent DM test set, the DCNN achieved a mean POA of 93.7% ± 6.9%, a mean Hdist of 3.80 ± 3.21 mm, and a mean AvgDist of 1.49 ± 1.62 mm comparing to 86.9% ± 16.0%, 7.18 ± 14.22 mm, and 3.98 ± 14.13 mm, respectively, by the baseline method.

CONCLUSION

In comparison to the TFO method, DCNN significantly improved the accuracy of pectoral muscle identification on mammograms (P < 0.05).

Deep Learning Approach for Assessment of Bladder Cancer Treatment Response

We compared the performance of different Deep learning-convolutional neural network (DL-CNN) models for bladder cancer treatment response assessment based on transfer learning by freezing different DL-CNN layers and varying the DL-CNN structure. Pre- and posttreatment computed tomography scans of 123 patients (cancers, 129; pre- and posttreatment cancer pairs, 158) undergoing chemotherapy were collected. After chemotherapy 33% of patients had T0 stage cancer (complete response). Regions of interest in pre- and posttreatment scans were extracted from the segmented lesions and combined into hybrid pre -post image pairs (h-ROIs). Training (pairs, 94; h-ROIs, 6209), validation (10 pairs) and test sets (54 pairs) were obtained. The DL-CNN consisted of 2 convolution (C1-C2), 2 locally connected (L3-L4), and 1 fully connected layers. The DL-CNN was trained with h-ROIs to classify cancers as fully responding (stage T0) or not fully responding to chemotherapy. Two radiologists provided lesion likelihood of being stage T0 posttreatment. The test area under the ROC curve (AUC) was 0.73 for T0 prediction by the base DL-CNN structure with randomly initialized weights. The base DL-CNN structure with pretrained weights and transfer learning (no frozen layers) achieved test AUC of 0.79. The test AUCs for 3 modified DL-CNN structures (different C1-C2 max pooling filter sizes, strides, and padding, with transfer learning) were 0.72, 0.86, and 0.69. For the base DL-CNN with (C1) frozen, (C1-C2) frozen, and (C1-C2-L3) frozen, the test AUCs were 0.81, 0.78, and 0.71, respectively. The radiologists' AUCs were 0.76 and 0.77. DL-CNN performed better with pretrained than randomly initialized weights.

U-Net based deep learning bladder segmentation in CT urography

OBJECTIVES

To develop a U-Net-based deep learning approach (U-DL) for bladder segmentation in computed tomography urography (CTU) as a part of a computer-assisted bladder cancer detection and treatment response assessment pipeline.

MATERIALS AND METHODS

A dataset of 173 cases including 81 cases in the training/validation set (42 masses, 21 with wall thickening, 18 normal bladders), and 92 cases in the test set (43 masses, 36 with wall thickening, 13 normal bladders) were used with Institutional Review Board approval. An experienced radiologist provided three-dimensional (3D) hand outlines for all cases as the reference standard. We previously developed a bladder segmentation method that used a deep learning convolution neural network and level sets (DCNN-LS) within a user-input bounding box. However, some cases with poor image quality or with advanced bladder cancer spreading into the neighboring organs caused inaccurate segmentation. We have newly developed an automated U-DL method to estimate a likelihood map of the bladder in CTU. The U-DL did not require a user-input box and the level sets for postprocessing. To identify the best model for this task, we compared the following models: (a) two-dimensional (2D) U-DL and 3D U-DL using 2D CT slices and 3D CT volumes, respectively, as input, (b) U-DLs using CT images of different resolutions as input, and (c) U-DLs with and without automated cropping of the bladder as an image preprocessing step. The segmentation accuracy relative to the reference standard was quantified by six measures: average volume intersection ratio (AVI), average percent volume error (AVE), average absolute volume error (AAVE), average minimum distance (AMD), average Hausdorff distance (AHD), and the average Jaccard index (AJI). As a baseline, the results from our previous DCNN-LS method were used.

RESULTS

In the test set, the best 2D U-DL model achieved AVI, AVE, AAVE, AMD, AHD, and AJI values of 93.4 ± 9.5%, -4.2 ± 14.2%, 9.2 ± 11.5%, 2.7 ± 2.5 mm, 9.7 ± 7.6 mm, 85.0 ± 11.3%, respectively, while the corresponding measures by the best 3D U-DL were 90.6 ± 11.9%, -2.3 ± 21.7%, 11.5 ± 18.5%, 3.1 ± 3.2 mm, 11.4 ± 10.0 mm, and 82.6 ± 14.2%, respectively. For comparison, the corresponding values obtained with the baseline method were 81.9 ± 12.1%, 10.2 ± 16.2%, 14.0 ± 13.0%, 3.6 ± 2.0 mm, 12.8 ± 6.1 mm, and 76.2 ± 11.8%, respectively, for the same test set. The improvement for all measures between the best U-DL and the DCNN-LS were statistically significant (P < 0.001).

CONCLUSION

Compared to a previous DCNN-LS method, which depended on a user-input bounding box, the U-DL provided more accurate bladder segmentation and was more automated than the previous approach.

Synthesizing mammogram from digital breast tomosynthesis

The purpose of this study is to develop a new method for generating synthesized mammogram (SM) from digital breast tomosynthesis (DBT) and to assess its potential as an adjunct to DBT. We first applied multiscale bilateral filtering to the reconstructed DBT slices to enhance the high-frequency features and reduce noise. A maximum intensity projection (MIP) image was then obtained from the high-frequency components of the DBT slices. A multiscale image fusion method was designed to combine the MIP image and the central DBT projection view into an SM and further enhance the high-frequency features. We conducted a pilot reader study to visually assess the image quality of SM in comparison to full field digital mammograms (FFDM). For each DBT craniocaudal or mediolateral view, a clinical FFDM of the corresponding view was retrospectively collected. Three MQSA radiologists, blinded to the pathological and other clinical information, independently interpreted the SM and the corresponding FFDM side by side marked with the lesion locations. The differences in the BI-RADS assessments of both MCs and masses between SM and FFDM did not achieve statistical significance for all three readers. The conspicuity of MCs on SM was superior to that on FFDM and the BI-RADS assessments of MCs were comparable while the conspicuity of masses on SM was degraded and interpretation on SM was less accurate than that on FFDM. The SM may be useful for efficient prescreening of MCs in DBT but the DBT should be used for detection and characterization of masses.

Deep-learning convolutional neural network: Inner and outer bladder wall segmentation in CT urography

PURPOSE

We are developing a computerized segmentation tool for the inner and outer bladder wall as a part of an image analysis pipeline for CT urography (CTU).

MATERIALS AND METHODS

A data set of 172 CTU cases was collected retrospectively with Institutional Review Board (IRB) approval. The data set was randomly split into two independent sets of training (81 cases) and testing (92 cases) which were manually outlined for both the inner and outer wall. We trained a deep-learning convolutional neural network (DL-CNN) to distinguish the bladder wall from the inside and outside of the bladder using neighborhood information. Approximately, 240 000 regions of interest (ROIs) of 16 × 16 pixels in size were extracted from regions in the training cases identified by the manually outlined inner and outer bladder walls to form a training set for the DL-CNN; half of the ROIs were selected to include the bladder wall and the other half were selected to exclude the bladder wall with some of these ROIs being inside the bladder and the rest outside the bladder entirely. The DL-CNN trained on these ROIs was applied to the cases in the test set slice-by-slice to generate a bladder wall likelihood map where the gray level of a given pixel represents the likelihood that a given pixel would belong to the bladder wall. We then used the DL-CNN likelihood map as an energy term in the energy equation of a cascaded level sets method to segment the inner and outer bladder wall. The DL-CNN segmentation with level sets was compared to the three-dimensional (3D) hand-segmented contours as a reference standard.

RESULTS

For the inner wall contour, the training set achieved the average volume intersection, average volume error, average absolute volume error, and average distance of 90.0 ± 8.7%, -4.2 ± 18.4%, 12.9 ± 13.9%, and 3.0 ± 1.6 mm, respectively. The corresponding values for the test set were 86.9 ± 9.6%, -8.3 ± 37.7%, 18.4 ± 33.8%, and 3.4 ± 1.8 mm, respectively. For the outer wall contour, the training set achieved the values of 93.7 ± 3.9%, -7.8 ± 11.4%, 10.3 ± 9.3%, and 3.0 ± 1.2 mm, respectively. The corresponding values for the test set were 87.5 ± 9.9%, -1.2 ± 20.8%, 11.9 ± 17.0%, and 3.5 ± 2.3 mm, respectively.

CONCLUSIONS

Our study demonstrates that DL-CNN-assisted level sets can effectively segment bladder walls from the inner bladder and outer structures despite a lack of consistent distinctions along the inner wall. However, even with the addition of level sets, the inner and outer walls may still be over-segmented and the DL-CNN-assisted level sets may incorrectly segment parts of the prostate that overlap with the outer bladder wall. The outer wall segmentation was improved compared to our previous method and the DL-CNN-assisted level sets were also able to segment the inner bladder wall with similar performance. This study shows the DL-CNN-assisted level set segmentation tool can effectively segment the inner and outer wall of the bladder.

Assessment of mammographic breast density after sleeve gastrectomy

BACKGROUND

Mammographic breast density (BD) is an independent risk factor for breast cancer. The effects of bariatric surgery on BD are unknown.

OBJECTIVES

To investigate BD changes after sleeve gastrectomy (SG).

SETTING

University hospital, United States.

METHODS

Fifty women with mammograms before and after SG performed from 2009 to 2015 were identified after excluding patients with a history of breast cancer, hormone replacement, and/or breast surgery. Patient age, menopausal status, co-morbidities, hemoglobin A1C, and body mass index were collected. Craniocaudal mammographic views before and after SG were interpreted by a blinded radiologist and analyzed by software to obtain breast imaging reporting and data system density categories, breast area, BD, and absolute dense breast area (ADA). Analyses were performed using χ², McNemar's test, t test, and linear regressions.

RESULTS

Radiologist interpretation revealed a significant increase in breast imaging reporting and data system B+C category (68% versus 54%; P = .0095) and BD (9.8 ± 7.4% versus 8.3 ± 6.4%; P = .0006) after SG. Software analyses showed a postoperative decrease in breast area (75,398.9 ± 22,941.2 versus 90,655.9 ± 25,621.0 pixels; P < .0001) and ADA (7287.1 ± 3951.3 versus 8204.6 ± 4769.9 pixels; P = .0314) with no significant change in BD. Reduction in ADA was accentuated in postmenopausal patients. Declining breast area was directly correlated with body mass index reduction (R² = .4495; P < 0.0001). Changes in breast rather than whole body adiposity better explained ADA reduction. Neither diabetes status nor changes in hemoglobin A1C correlated with changes in ADA.

CONCLUSIONS

ADA decreases after SG, particularly in postmenopausal patients. Software-generated ADA may be more accurate than radiologist-estimated BD or breast imaging reporting and data system for capturing changes in dense breast tissue after SG.

Evolutionary pruning of transfer learned deep convolutional neural network for breast cancer diagnosis in digital breast tomosynthesis

Deep learning models are highly parameterized, resulting in difficulty in inference and transfer learning for image recognition tasks. In this work, we propose a layered pathway evolution method to compress a deep convolutional neural network (DCNN) for classification of masses in digital breast tomosynthesis (DBT). The objective is to prune the number of tunable parameters while preserving the classification accuracy. In the first stage transfer learning, 19 632 augmented regions-of-interest (ROIs) from 2454 mass lesions on mammograms were used to train a pre-trained DCNN on ImageNet. In the second stage transfer learning, the DCNN was used as a feature extractor followed by feature selection and random forest classification. The pathway evolution was performed using genetic algorithm in an iterative approach with tournament selection driven by count-preserving crossover and mutation. The second stage was trained with 9120 DBT ROIs from 228 mass lesions using leave-one-case-out cross-validation. The DCNN was reduced by 87% in the number of neurons, 34% in the number of parameters, and 95% in the number of multiply-and-add operations required in the convolutional layers. The test AUC on 89 mass lesions from 94 independent DBT cases before and after pruning were 0.88 and 0.90, respectively, and the difference was not statistically significant (p  >  0.05). The proposed DCNN compression approach can reduce the number of required operations by 95% while maintaining the classification performance. The approach can be extended to other deep neural networks and imaging tasks where transfer learning is appropriate.

Semi-automated pulmonary nodule interval segmentation using the NLST data

PURPOSE

To study the variability in volume change estimates of pulmonary nodules due to segmentation approaches used across several algorithms and to evaluate these effects on the ability to predict nodule malignancy.

METHODS

We obtained 100 patient image datasets from the National Lung Screening Trial (NLST) that had a nodule detected on each of two consecutive low dose computed tomography (LDCT) scans, with an equal proportion of malignant and benign cases (50 malignant, 50 benign). Information about the nodule location for the cases was provided by a screen capture with a bounding box and its axial location was indicated. Five participating quantitative imaging network (QIN) institutions performed nodule segmentation using their preferred semi-automated algorithms with no manual correction; teams were allowed to provide additional manually corrected segmentations (analyzed separately). The teams were asked to provide segmentation masks for each nodule at both time points. From these masks, the volume was estimated for the nodule at each time point; the change in volume (absolute and percent change) across time points was estimated as well. We used the concordance correlation coefficient (CCC) to compare the similarity of computed nodule volumes (absolute and percent change) across algorithms. We used Logistic regression model on the change in volume (absolute change and percent change) of the nodules to predict the malignancy status, the area under the receiver operating characteristic curve (AUROC) and confidence intervals were reported. Because the size of nodules was expected to have a substantial effect on segmentation variability, analysis of change in volumes was stratified by lesion size, where lesions were grouped into those with a longest diameter of <8 mm and those with longest diameter ≥ 8 mm.

RESULTS

We find that segmentation of the nodules shows substantial variability across algorithms, with the CCC ranging from 0.56 to 0.95 for change in volume (percent change in volume range was [0.15 to 0.86]) across the nodules. When examining nodules based on their longest diameter, we find the CCC had higher values for large nodules with a range of [0.54 to 0.93] among the algorithms, while percent change in volume was [0.3 to 0.95]. Compared to that of smaller nodules which had a range of [-0.0038 to 0.69] and percent change in volume was [-0.039 to 0.92]. The malignancy prediction results showed fairly consistent results across the institutions, the AUC using change in volume ranged from 0.65 to 0.89 (Percent change in volume was 0.64 to 0.86) for entire nodule range. Prediction improves for large nodule range (≥ 8 mm) with AUC range 0.75 to 0.90 (percent change in volume was 0.74 to 0.92). Compared to smaller nodule range (<8 mm) with AUC range 0.57 to 0.78 (percent change in volume was 0.59 to 0.77).

CONCLUSIONS

We find there is a fairly high concordance in the size measurements for larger nodules (≥8 mm) than the lower sizes (<8 mm) across algorithms. We find the change in nodule volume (absolute and percent change) were consistent predictors of malignancy across institutions, despite using different segmentation algorithms. Using volume change estimates without corrections shows slightly lower predictability (for two teams).

Computer-aided assessment of breast density: comparison of supervised deep learning and feature-based statistical learning

Breast density is one of the most significant factors that is associated with cancer risk. In this study, our purpose was to develop a supervised deep learning approach for automated estimation of percentage density (PD) on digital mammograms (DMs). The input 'for processing' DMs was first log-transformed, enhanced by a multi-resolution preprocessing scheme, and subsampled to a pixel size of 800 µm  ×  800 µm from 100 µm  ×  100 µm. A deep convolutional neural network (DCNN) was trained to estimate a probability map of breast density (PMD) by using a domain adaptation resampling method. The PD was estimated as the ratio of the dense area to the breast area based on the PMD. The DCNN approach was compared to a feature-based statistical learning approach. Gray level, texture and morphological features were extracted and a least absolute shrinkage and selection operator was used to combine the features into a feature-based PMD. With approval of the Institutional Review Board, we retrospectively collected a training set of 478 DMs and an independent test set of 183 DMs from patient files in our institution. Two experienced mammography quality standards act radiologists interactively segmented PD as the reference standard. Ten-fold cross-validation was used for model selection and evaluation with the training set. With cross-validation, DCNN obtained a Dice's coefficient (DC) of 0.79  ±  0.13 and Pearson's correlation (r) of 0.97, whereas feature-based learning obtained DC  =  0.72  ±  0.18 and r  =  0.85. For the independent test set, DCNN achieved DC  =  0.76  ±  0.09 and r  =  0.94, while feature-based learning achieved DC  =  0.62  ±  0.21 and r  =  0.75. Our DCNN approach was significantly better and more robust than the feature-based learning approach for automated PD estimation on DMs, demonstrating its potential use for automated density reporting as well as for model-based risk prediction.

Detector Blur and Correlated Noise Modeling for Digital Breast Tomosynthesis Reconstruction

This paper describes a new image reconstruction method for digital breast tomosynthesis (DBT). The new method incorporates detector blur into the forward model. The detector blur in DBT causes correlation in the measurement noise. By making a few approximations that are reasonable for breast imaging, we formulated a regularized quadratic optimization problem with a data-fit term that incorporates models for detector blur and correlated noise (DBCN). We derived a computationally efficient separable quadratic surrogate (SQS) algorithm to solve the optimization problem that has a non-diagonal noise covariance matrix. We evaluated the SQS-DBCN method by reconstructing DBT scans of breast phantoms and human subjects. The contrast-to-noise ratio and sharpness of microcalcifications were analyzed and compared with those by the simultaneous algebraic reconstruction technique. The quality of soft tissue lesions and parenchymal patterns was examined. The results demonstrate the potential to improve the image quality of reconstructed DBT images by incorporating the system physics model. This paper is a first step toward model-based iterative reconstruction for DBT.

Multi-task transfer learning deep convolutional neural network: application to computer-aided diagnosis of breast cancer on mammograms

Transfer learning in deep convolutional neural networks (DCNNs) is an important step in its application to medical imaging tasks. We propose a multi-task transfer learning DCNN with the aim of translating the 'knowledge' learned from non-medical images to medical diagnostic tasks through supervised training and increasing the generalization capabilities of DCNNs by simultaneously learning auxiliary tasks. We studied this approach in an important application: classification of malignant and benign breast masses. With Institutional Review Board (IRB) approval, digitized screen-film mammograms (SFMs) and digital mammograms (DMs) were collected from our patient files and additional SFMs were obtained from the Digital Database for Screening Mammography. The data set consisted of 2242 views with 2454 masses (1057 malignant, 1397 benign). In single-task transfer learning, the DCNN was trained and tested on SFMs. In multi-task transfer learning, SFMs and DMs were used to train the DCNN, which was then tested on SFMs. N-fold cross-validation with the training set was used for training and parameter optimization. On the independent test set, the multi-task transfer learning DCNN was found to have significantly (p  =  0.007) higher performance compared to the single-task transfer learning DCNN. This study demonstrates that multi-task transfer learning may be an effective approach for training DCNN in medical imaging applications when training samples from a single modality are limited.

Improving image quality for digital breast tomosynthesis: an automated detection and diffusion-based method for metal artifact reduction

In digital breast tomosynthesis (DBT), the high-attenuation metallic clips marking a previous biopsy site in the breast cause errors in the estimation of attenuation along the ray paths intersecting the markers during reconstruction, which result in interplane and inplane artifacts obscuring the visibility of subtle lesions. We proposed a new metal artifact reduction (MAR) method to improve image quality. Our method uses automatic detection and segmentation to generate a marker location map for each projection (PV). A voting technique based on the geometric correlation among different PVs is designed to reduce false positives (FPs) and to label the pixels on the PVs and the voxels in the imaged volume that represent the location and shape of the markers. An iterative diffusion method replaces the labeled pixels on the PVs with estimated tissue intensity from the neighboring regions while preserving the original pixel values in the neighboring regions. The inpainted PVs are then used for DBT reconstruction. The markers are repainted on the reconstructed DBT slices for radiologists' information. The MAR method is independent of reconstruction techniques or acquisition geometry. For the training set, the method achieved 100% success rate with one FP in 19 views. For the test set, the success rate by view was 97.2% for core biopsy microclips and 66.7% for clusters of large post-lumpectomy markers with a total of 10 FPs in 58 views. All FPs were large dense benign calcifications that also generated artifacts if they were not corrected by MAR. For the views with successful detection, the metal artifacts were reduced to a level that was not visually apparent in the reconstructed slices. The visibility of breast lesions obscured by the reconstruction artifacts from the metallic markers was restored.

Urinary bladder cancer staging in CT urography using machine learning

PURPOSE

To evaluate the feasibility of using an objective computer-aided system to assess bladder cancer stage in CT Urography (CTU).

MATERIALS AND METHODS

A dataset consisting of 84 bladder cancer lesions from 76 CTU cases was used to develop the computerized system for bladder cancer staging based on machine learning approaches. The cases were grouped into two classes based on pathological stage ≥ T2 or below T2, which is the decision threshold for neoadjuvant chemotherapy treatment clinically. There were 43 cancers below stage T2 and 41 cancers at stage T2 or above. All 84 lesions were automatically segmented using our previously developed auto-initialized cascaded level sets (AI-CALS) method. Morphological and texture features were extracted. The features were divided into subspaces of morphological features only, texture features only, and a combined set of both morphological and texture features. The dataset was split into Set 1 and Set 2 for two-fold cross-validation. Stepwise feature selection was used to select the most effective features. A linear discriminant analysis (LDA), a neural network (NN), a support vector machine (SVM), and a random forest (RAF) classifier were used to combine the features into a single score. The classification accuracy of the four classifiers was compared using the area under the receiver operating characteristic (ROC) curve (A_z ).

RESULTS

Based on the texture features only, the LDA classifier achieved a test A_z of 0.91 on Set 1 and a test A_z of 0.88 on Set 2. The test A_z of the NN classifier for Set 1 and Set 2 were 0.89 and 0.92, respectively. The SVM classifier achieved test A_z of 0.91 on Set 1 and test A_z of 0.89 on Set 2. The test A_z of the RAF classifier for Set 1 and Set 2 was 0.89 and 0.97, respectively. The morphological features alone, the texture features alone, and the combined feature set achieved comparable classification performance.

CONCLUSION

The predictive model developed in this study shows promise as a classification tool for stratifying bladder cancer into two staging categories: greater than or equal to stage T2 and below stage T2.

Segmented separable footprint projector for digital breast tomosynthesis and its application for subpixel reconstruction

PURPOSE

Digital forward and back projectors play a significant role in iterative image reconstruction. The accuracy of the projector affects the quality of the reconstructed images. Digital breast tomosynthesis (DBT) often uses the ray-tracing (RT) projector that ignores finite detector element size. This paper proposes a modified version of the separable footprint (SF) projector, called the segmented separable footprint (SG) projector, that calculates efficiently the Radon transform mean value over each detector element. The SG projector is specifically designed for DBT reconstruction because of the large height-to-width ratio of the voxels generally used in DBT. This study evaluates the effectiveness of the SG projector in reducing projection error and improving DBT reconstruction quality.

METHODS

We quantitatively compared the projection error of the RT and the SG projector at different locations and their performance in regular and subpixel DBT reconstruction. Subpixel reconstructions used finer voxels in the imaged volume than the detector pixel size. Subpixel reconstruction with RT projector uses interpolated projection views as input to provide adequate coverage of the finer voxel grid with the traced rays. Subpixel reconstruction with the SG projector, however, uses the measured projection views without interpolation. We simulated DBT projections of a test phantom using CatSim (GE Global Research, Niskayuna, NY) under idealized imaging conditions without noise and blur, to analyze the effects of the projectors and subpixel reconstruction without other image degrading factors. The phantom contained an array of horizontal and vertical line pair patterns (1 to 9.5 line pairs/mm) and pairs of closely spaced spheres (diameters 0.053 to 0.5 mm) embedded at the mid-plane of a 5-cm-thick breast tissue-equivalent uniform volume. The images were reconstructed with regular simultaneous algebraic reconstruction technique (SART) and subpixel SART using different projectors. The resolution and contrast of the test objects in the reconstructed images and the computation times were compared under different reconstruction conditions.

RESULTS

The SG projector reduced the projector error by 1 to 2 orders of magnitude at most locations. In the worst case, the SG projector still reduced the projection error by about 50%. In the DBT reconstructed slices parallel to the detector plane, the SG projector not only increased the contrast of the line pairs and spheres but also produced more smooth and continuous reconstructed images, whereas the discrete and sparse nature of the RT projector caused artifacts appearing as patterned noise. For subpixel reconstruction, the SG projector significantly increased object contrast and computation speed, especially for high subpixel ratios, compared with the RT projector implemented with accelerated Siddon's algorithm. The difference in the depth resolution among the projectors is negligible under the conditions studied. Our results also demonstrated that subpixel reconstruction can improve the spatial resolution of the reconstructed images, and can exceed the Nyquist limit of the detector under some conditions.

CONCLUSIONS

The SG projector was more accurate and faster than the RT projector. The SG projector also substantially reduced computation time and improved the image quality for the tomosynthesized images with and without subpixel reconstruction.

Mass detection in digital breast tomosynthesis: Deep convolutional neural network with transfer learning from mammography

PURPOSE

Develop a computer-aided detection (CAD) system for masses in digital breast tomosynthesis (DBT) volume using a deep convolutional neural network (DCNN) with transfer learning from mammograms.

METHODS

A data set containing 2282 digitized film and digital mammograms and 324 DBT volumes were collected with IRB approval. The mass of interest on the images was marked by an experienced breast radiologist as reference standard. The data set was partitioned into a training set (2282 mammograms with 2461 masses and 230 DBT views with 228 masses) and an independent test set (94 DBT views with 89 masses). For DCNN training, the region of interest (ROI) containing the mass (true positive) was extracted from each image. False positive (FP) ROIs were identified at prescreening by their previously developed CAD systems. After data augmentation, a total of 45 072 mammographic ROIs and 37 450 DBT ROIs were obtained. Data normalization and reduction of non-uniformity in the ROIs across heterogeneous data was achieved using a background correction method applied to each ROI. A DCNN with four convolutional layers and three fully connected (FC) layers was first trained on the mammography data. Jittering and dropout techniques were used to reduce overfitting. After training with the mammographic ROIs, all weights in the first three convolutional layers were frozen, and only the last convolution layer and the FC layers were randomly initialized again and trained using the DBT training ROIs. The authors compared the performances of two CAD systems for mass detection in DBT: one used the DCNN-based approach and the other used their previously developed feature-based approach for FP reduction. The prescreening stage was identical in both systems, passing the same set of mass candidates to the FP reduction stage. For the feature-based CAD system, 3D clustering and active contour method was used for segmentation; morphological, gray level, and texture features were extracted and merged with a linear discriminant classifier to score the detected masses. For the DCNN-based CAD system, ROIs from five consecutive slices centered at each candidate were passed through the trained DCNN and a mass likelihood score was generated. The performances of the CAD systems were evaluated using free-response ROC curves and the performance difference was analyzed using a non-parametric method.

RESULTS

Before transfer learning, the DCNN trained only on mammograms with an AUC of 0.99 classified DBT masses with an AUC of 0.81 in the DBT training set. After transfer learning with DBT, the AUC improved to 0.90. For breast-based CAD detection in the test set, the sensitivity for the feature-based and the DCNN-based CAD systems was 83% and 91%, respectively, at 1 FP/DBT volume. The difference between the performances for the two systems was statistically significant (p-value < 0.05).

CONCLUSIONS

The image patterns learned from the mammograms were transferred to the mass detection on DBT slices through the DCNN. This study demonstrated that large data sets collected from mammography are useful for developing new CAD systems for DBT, alleviating the problem and effort of collecting entirely new large data sets for the new modality.