Classification of microcalcification clusters in digital breast tomosynthesis using ensemble convolutional neural network

Development of a Dual-Plane MRI-Based Deep Learning Model to Assess the 1-Year Postoperative Outcomes in Lumbar Disc Herniation After Tubular Microdiscectomy

BACKGROUND

Tubular microdiscectomy (TMD) is a treatment for lumbar disc herniation (LDH). Although the combination of MRI and deep learning (DL) has shown promise, its application in evaluating postoperative outcomes in TMD has not been fully explored.

PURPOSE/HYPOTHESIS

To evaluate whether integrating preoperative dual-plane MRI-based DL features with clinical features can assess 1-year outcomes in TMD for LDH.

STUDY TYPE

Retrospective.

POPULATION/SUBJECTS

The study involved 548 patients who underwent TMD between January 2016 and January 2021. Training set (N = 305, mean age 51.85 ± 13.84 years, 56.4% male). Internal validation set (N = 131, mean age 51.85 ± 13.84 years, 54.2% male). External validation set (N = 112, mean age 51.54 ± 14.43 years, 50.9% male).

FIELD STRENGTH/SEQUENCE

3 T MRI with sagittal and transverse T2-weighted sequences (Fast Spin Echo).

ASSESSMENT

Ground truth labels were based on improvement rate in 1-year Japanese Orthopaedic Association (JOA) scores. Information on 42 preoperative clinical features was collected. The largest protrusions were identified from T2 MRI by three clinicians and were used to train deep learning models (ResNet50, ResNet101, and ResNet152) to extract DL features. After feature selection, three models were built, namely, clinical, DL, and combined models.

STATISTICAL TESTS

Chi-square or Fisher's exact tests was used for group comparisons. Quantitative differences were analyzed using the t-test or Mann-Whitney U test. P-values <0.05 were considered significant. Models were validated on internal and external datasets using metrics such as the area under the curve (AUC).

RESULTS

The AUCs of the clinical models achieved 0.806 (internal) and 0.779 (external). ResNet152 performed best in three DL models, with AUCs of 0.858 (internal) and 0.834 (external). The combined model achieved AUCs of 0.889 (internal) and 0.857 (external).

DATA CONCLUSION

A model combining preoperative dual-plane MRI DL features and clinical features can assess 1-year outcomes of TMD for LDH.

EVIDENCE LEVEL

4 TECHNICAL EFFICACY: Stage 2.

Multicenter study on predicting postoperative upper limb muscle strength improvement in cervical spinal cord injury patients using radiomics and deep learning

Cervical spinal cord injury is often catastrophic, frequently leading to irreversible impairment. MRI has become the gold standard for evaluating spinal cord injuries (SCI). Our study aimed to assess the accuracy of a radiomics approach, based on machine learning and utilizing conventional MRI, in predicting the prognosis of patients with SCI. In a retrospective analysis of 82 SCI patients from three hospitals, we categorized them into good (n = 49) and poor (n = 33) prognosis groups. Preoperative T2-weighted MRI images were segmented using 3D-Region of Interest (ROI) techniques, and both radiomic and deep transfer learning features were extracted. These features were normalized using Z-score and harmonized via ComBat. Feature selection was performed using a greedy algorithm and Least absolute shrinkage and selection operator (LASSO), and others, followed by the calculation of radiomics scores through linear regression. Machine learning was then used to identify the most predictive radiomic features. Model performance was evaluated by analyzing the area under the curve (AUC) and other indicators.Univariate analysis indicated that the demographic characteristics of cervical spinal cord injury were not statistically significant. In the test dataset, the random forest (RF) combined with radiomics and ResNet34 demonstrated better performance, with an accuracy of 0.800 and an AUC of 0.893.Using MRI, deep learning-based radiomics signals show promise in evaluating and predicting the postoperative prognosis of these patients.

A Radiomic and Clinical Data-Based Risk Model for Malignancy Prediction of Breast BI-RADS 4A Microcalcifications

BACKGROUND

Mammography is the gold standard technique for early breast cancer screening, but it has a limited specificity for microcalcifications. Radiomics represents a promising tool for enhancing lesion risk stratification. This study aims to evaluate the reliability of radiomics in combination with clinical data to classify benign and malignant microcalcifications, potentially enhancing the standard radiological assessment and reducing the need for biopsies.

MATERIALS AND METHODS

This study retrospectively analyzed patients with BI-RADS 4A microcalcifications who underwent mammography (MX) and vacuum-assisted breast biopsy (VABB) at our center from January 2019 to February 2023. About 104 radiomics features were extracted from a region of interest, manually defined on images. Clinical data from each patient were collected. Using the Tyrer-Cuzick Model, we classified patients according to the risk of developing breast cancer. Two logistic regression models, using clinical and radiomics data were trained to predict the pathological classification of breast calcifications.

RESULTS

A total of 167 calcification groups were included in the study. The final dataset was made of 14 radiomics features. The radiomics model alone achieved an AUC of 0.72 (95% CI, 0.61-0.33) while the model trained on clinical and radiomics features obtained AUC values of 0.81 (95% CI, 0.69-0.92).

CONCLUSIONS

Our findings suggest that the integration of clinical data with radiomics has the potential to reduce unnecessary biopsies for BI-RADS 4A microcalcifications, leading to more targeted and personalized patient care.

Artificial intelligence for breast cancer detection and its health technology assessment: A scoping review

BACKGROUND

Recent healthcare advancements highlight the potential of Artificial Intelligence (AI) - and especially, among its subfields, Machine Learning (ML) - in enhancing Breast Cancer (BC) clinical care, leading to improved patient outcomes and increased radiologists' efficiency. While medical imaging techniques have significantly contributed to BC detection and diagnosis, their synergy with AI algorithms has consistently demonstrated superior diagnostic accuracy, reduced False Positives (FPs), and enabled personalized treatment strategies. Despite the burgeoning enthusiasm for leveraging AI for early and effective BC clinical care, its widespread integration into clinical practice is yet to be realized, and the evaluation of AI-based health technologies in terms of health and economic outcomes remains an ongoing endeavor.

OBJECTIVES

This scoping review aims to investigate AI (and especially ML) applications that have been implemented and evaluated across diverse clinical tasks or decisions in breast imaging and to explore the current state of evidence concerning the assessment of AI-based technologies for BC clinical care within the context of Health Technology Assessment (HTA).

METHODS

We conducted a systematic literature search following the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) checklist in PubMed and Scopus to identify relevant studies on AI (and particularly ML) applications in BC detection and diagnosis. We limited our search to studies published from January 2015 to October 2023. The Minimum Information about CLinical Artificial Intelligence Modeling (MI-CLAIM) checklist was used to assess the quality of AI algorithms development, evaluation, and reporting quality in the reviewed articles. The HTA Core Model® was also used to analyze the comprehensiveness, robustness, and reliability of the reported results and evidence in AI-systems' evaluations to ensure rigorous assessment of AI systems' utility and cost-effectiveness in clinical practice.

RESULTS

Of the 1652 initially identified articles, 104 were deemed eligible for inclusion in the review. Most studies examined the clinical effectiveness of AI-based systems (78.84%, n= 82), with one study focusing on safety in clinical settings, and 13.46% (n=14) focusing on patients' benefits. Of the studies, 31.73% (n=33) were ethically approved to be carried out in clinical practice, whereas 25% (n=26) evaluated AI systems legally approved for clinical use. Notably, none of the studies addressed the organizational implications of AI systems in clinical practice. Of the 104 studies, only two of them focused on cost-effectiveness analysis, and were analyzed separately. The average percentage scores for the first 102 AI-based studies' quality assessment based on the MI-CLAIM checklist criteria were 84.12%, 83.92%, 83.98%, 74.51%, and 14.7% for study design, data and optimization, model performance, model examination, and reproducibility, respectively. Notably, 20.59% (n=21) of these studies relied on large-scale representative real-world breast screening datasets, with only 10.78% (n =11) studies demonstrating the robustness and generalizability of the evaluated AI systems.

CONCLUSION

In bridging the gap between cutting-edge developments and seamless integration of AI systems into clinical workflows, persistent challenges encompass data quality and availability, ethical and legal considerations, robustness and trustworthiness, scalability, and alignment with existing radiologists' workflow. These hurdles impede the synthesis of comprehensive, robust, and reliable evidence to substantiate these systems' clinical utility, relevance, and cost-effectiveness in real-world clinical workflows. Consequently, evaluating AI-based health technologies through established HTA methodologies becomes complicated. We also highlight potential significant influences on AI systems' effectiveness of various factors, such as operational dynamics, organizational structure, the application context of AI systems, and practices in breast screening or examination reading of AI support tools in radiology. Furthermore, we emphasize substantial reciprocal influences on decision-making processes between AI systems and radiologists. Thus, we advocate for an adapted assessment framework specifically designed to address these potential influences on AI systems' effectiveness, mainly addressing system-level transformative implications for AI systems rather than focusing solely on technical performance and task-level evaluations.

Imaging phenotype evaluation from digital breast tomosynthesis data: A preliminary study

BACKGROUND

Digital breast tomosynthesis (DBT) has been widely adopted as a supplemental imaging modality for diagnostic evaluation of breast cancer and confirmation studies. In this study, a deep learning-based method for characterizing breast tissue patterns in DBT data is presented.

METHODS

A set of 5388 2D image patches was produced from 230 right mediolateral oblique, 259 left mediolateral oblique, 18 right craniocaudal, and 15 left craniocaudal single-breast DBT studies, using slice-wise annotations of abnormalities and normal tissue. We implemented a patch classifier to predict samples according to two differing scenarios and train it using the patch dataset. First, tissue samples were classified into the following classes: malignant, benign, and normal breast tissue. Second, tissue samples were classified into the following classes: malignant mass, benign mass, benign architectural distortion, malignant architectural distortion, and normal breast tissue. We employed transfer learning and initialized the model base layers with existing pre-trained weights obtained from Globally-Aware Multiple Instance Classifier.

RESULTS

High class-wise recall values of 0.8906, 0.8541 and 0.7345 and specificities 0.9558, 0.9575 and 0.8830 were obtained for normal, benign, and malignant classification, respectively. More intricate classification yielded class-wise recall values of 0.8708, 0.8299, 0.9444 and 0.5723 and specificities 0.9406, 0.9833, 0.8943 and 0.9652 for benign mass, normal, malignant architectural distortion, and malignant mass, respectively. However, benign architectural distortion was confused with benign mass and malignant architectural distortion.

CONCLUSIONS

Combining the proposed phenotype classifier with the commonly used malignant-benign-normal classification enables a more detailed assessment of digital breast tomosynthesis images.

Hybrid Feature Mammogram Analysis: Detecting and Localizing Microcalcifications Combining Gabor, Prewitt, GLCM Features, and Top Hat Filtering Enhanced with CNN Architecture

Breast cancer is a prevalent malignancy characterized by the uncontrolled growth of glandular epithelial cells, which can metastasize through the blood and lymphatic systems. Microcalcifications, small calcium deposits within breast tissue, are critical markers for early detection of breast cancer, especially in non-palpable carcinomas. These microcalcifications, appearing as small white spots on mammograms, are challenging to identify due to potential confusion with other tissues. This study hypothesizes that a hybrid feature extraction approach combined with Convolutional Neural Networks (CNNs) can significantly enhance the detection and localization of microcalcifications in mammograms. The proposed algorithm employs Gabor, Prewitt, and Gray Level Co-occurrence Matrix (GLCM) kernels for feature extraction. These features are input to a CNN architecture designed with maxpooling layers, Rectified Linear Unit (ReLU) activation functions, and a sigmoid response for binary classification. Additionally, the Top Hat filter is used for precise localization of microcalcifications. The preprocessing stage includes enhancing contrast using the Volume of Interest Look-Up Table (VOI LUT) technique and segmenting regions of interest. The CNN architecture comprises three convolutional layers, three ReLU layers, and three maxpooling layers. The training was conducted using a balanced dataset of digital mammograms, with the Adam optimizer and binary cross-entropy loss function. Our method achieved an accuracy of 89.56%, a sensitivity of 82.14%, and a specificity of 91.47%, outperforming related works, which typically report accuracies around 85-87% and sensitivities between 76 and 81%. These results underscore the potential of combining traditional feature extraction techniques with deep learning models to improve the detection and localization of microcalcifications. This system may serve as an auxiliary tool for radiologists, enhancing early detection capabilities and potentially reducing diagnostic errors in mass screening programs.

Diagnosis of COVID-19 with simultaneous accurate prediction of cardiac abnormalities from chest computed tomographic images

COVID-19 has potential consequences on the pulmonary and cardiovascular health of millions of infected people worldwide. Chest computed tomographic (CT) imaging has remained the first line of diagnosis for individuals infected with SARS-CoV-2. However, differentiating COVID-19 from other types of pneumonia and predicting associated cardiovascular complications from the same chest-CT images have remained challenging. In this study, we have first used transfer learning method to distinguish COVID-19 from other pneumonia and healthy cases with 99.2% accuracy. Next, we have developed another CNN-based deep learning approach to automatically predict the risk of cardiovascular disease (CVD) in COVID-19 patients compared to the normal subjects with 97.97% accuracy. Our model was further validated against cardiac CT-based markers including cardiac thoracic ratio (CTR), pulmonary artery to aorta ratio (PA/A), and presence of calcified plaque. Thus, we successfully demonstrate that CT-based deep learning algorithms can be employed as a dual screening diagnostic tool to diagnose COVID-19 and differentiate it from other pneumonia, and also predicts CVD risk associated with COVID-19 infection.

CAPNet: Context attention pyramid network for computer-aided detection of microcalcification clusters in digital breast tomosynthesis

BACKGROUND AND OBJECTIVE

Computer-aided detection (CADe) of microcalcification clusters (MCs) in digital breast tomosynthesis (DBT) is crucial in the early diagnosis of breast cancer. Although convolutional neural network (CNN)-based detection models have achieved excellent performance in medical lesion detection, they are subject to some limitations in MC detection: 1) Most existing models employ the feature pyramid network (FPN) for multi-scale object detection; however, the rough feature sharing between adjacent layers in the FPN may limit the detection ability for small and low-contrast MCs; and 2) the MCs region only accounts for a small part of the annotation box, so the features extracted indiscriminately within the whole box may easily be affected by the background. In this paper, we develop a novel CNN-based CADe method to alleviate the impacts of the above limitations for the accurate and rapid detection of MCs in DBT.

METHODS

The proposed method has two parts: a novel context attention pyramid network (CAPNet) for intra-layer MC detection in two-dimensional (2D) slices and a three-dimensional (3D) aggregation procedure for aggregating 2D intra-layer MCs into a 3D result according to their connectivity in 3D space. The proposed CAPNet is based on an anchor-free and one-stage detection architecture and contains a context feature selection fusion (CFSF) module and a microcalcification response (MCR) branch. The CFSF module can efficiently enrich shallow layers' features by the complementary selection of local context features, aiming to reduce the missed detection of small and low-contrast MCs. The MCR branch is a one-layer branch parallel to the classification branch, which can alleviate the influence of the background region within the annotation box on feature extraction and enhance the ability of the model to distinguish MCs from normal breast tissue.

RESULTS

We performed a comparison experiment on an in-house clinical dataset with 648 DBT volumes, and the proposed method achieved impressive performance with a sensitivity of 91.56 % at 1 false positive per DBT volume (FPs/volume) and 93.51 % at 2 FPs/volume, outperforming other representative detection models.

CONCLUSIONS

The experimental results indicate that the proposed method is effective in the detection of MCs in DBT. This method can provide objective, accurate, and quick diagnostic suggestions for radiologists, presenting potential clinical value for early breast cancer screening.

AI-based computer-aided diagnostic system of chest digital tomography synthesis: Demonstrating comparative advantage with X-ray-based AI systems

BACKGROUND

Compared with chest X-ray (CXR) imaging, which is a single image projected from the front of the patient, chest digital tomosynthesis (CDTS) imaging can be more advantageous for lung lesion detection because it acquires multiple images projected from multiple angles of the patient. Various clinical comparative analysis and verification studies have been reported to demonstrate this, but there is no artificial intelligence (AI)-based comparative analysis studies. Existing AI-based computer-aided detection (CAD) systems for lung lesion diagnosis have been developed mainly based on CXR images; however, CAD-based on CDTS, which uses multi-angle images of patients in various directions, has not been proposed and verified for its usefulness compared to CXR-based counterparts.

BACKGROUND AND OBJECTIVE

This study develops and tests a CDTS-based AI CAD system to detect lung lesions to demonstrate performance improvements compared to CXR-based AI CAD.

METHODS

We used multiple (e.g., five) projection images as input for the CDTS-based AI model and a single-projection image as input for the CXR-based AI model to compare and evaluate the performance between models. Multiple/single projection input images were obtained by virtual projection on the three-dimensional (3D) stack of computed tomography (CT) slices of each patient's lungs from which the bed area was removed. These multiple images result from shooting from the front and left and right 30/60∘. The projected image captured from the front was used as the input for the CXR-based AI model. The CDTS-based AI model used all five projected images. The proposed CDTS-based AI model consisted of five AI models that received images in each of the five directions, and obtained the final prediction result through an ensemble of five models. Each model used WideResNet-50. To train and evaluate CXR- and CDTS-based AI models, 500 healthy data, 206 tuberculosis data, and 242 pneumonia data were used, and three three-fold cross-validation was applied.

RESULTS

The proposed CDTS-based AI CAD system yielded sensitivities of 0.782 and 0.785 and accuracies of 0.895 and 0.837 for the (binary classification) performance of detecting tuberculosis and pneumonia, respectively, against normal subjects. These results show higher performance than the sensitivity of 0.728 and 0.698 and accuracies of 0.874 and 0.826 for detecting tuberculosis and pneumonia through the CXR-based AI CAD, which only uses a single projection image in the frontal direction. We found that CDTS-based AI CAD improved the sensitivity of tuberculosis and pneumonia by 5.4% and 8.7% respectively, compared to CXR-based AI CAD without loss of accuracy.

CONCLUSIONS

This study comparatively proves that CDTS-based AI CAD technology can improve performance more than CXR. These results suggest that we can enhance the clinical application of CDTS. Our code is available at https://github.com/kskim-phd/CDTS-CAD-P.

Avoiding Tissue Overlap in 2D Images: Single-Slice DBT Classification Using Convolutional Neural Networks

Breast cancer was the most diagnosed cancer around the world in 2020. Screening programs, based on mammography, aim to achieve early diagnosis which is of extreme importance when it comes to cancer. There are several flaws associated with mammography, with one of the most important being tissue overlapping that can result in both lesion masking and fake-lesion appearance. To overcome this, digital breast tomosynthesis takes images (slices) at different angles that are later reconstructed into a 3D image. Having in mind that the slices are planar images where tissue overlapping does not occur, the goal of the work done here was to develop a deep learning model that could, based on the said slices, classify lesions as benign or malignant. The developed model was based on the work done by Muduli et. al, with a slight change in the fully connected layers and in the regularization done. In total, 77 DBT volumes-39 benign and 38 malignant-were available. From each volume, nine slices were taken, one where the lesion was most visible and four above/below. To increase the quantity and the variability of the data, common data augmentation techniques (rotation, translation, mirroring) were applied to the original images three times. Therefore, 2772 images were used for training. Data augmentation techniques were then applied two more times-one set used for validation and one set used for testing. Our model achieved, on the testing set, an accuracy of 93.2% while the values of sensitivity, specificity, precision, F1-score, and Cohen's kappa were 92%, 94%, 94%, 94%, and 0.86, respectively. Given these results, the work done here suggests that the use of single-slice DBT can compare to state-of-the-art studies and gives a hint that with more data, better augmentation techniques and the use of transfer learning might overcome the use of mammograms in this type of studies.

Predictive performance of radiomic models based on features extracted from pretrained deep networks

OBJECTIVES

In radiomics, generic texture and morphological features are often used for modeling. Recently, features extracted from pretrained deep networks have been used as an alternative. However, extracting deep features involves several decisions, and it is unclear how these affect the resulting models. Therefore, in this study, we considered the influence of such choices on the predictive performance.

METHODS

On ten publicly available radiomic datasets, models were trained using feature sets that differed in terms of the utilized network architecture, the layer of feature extraction, the used set of slices, the use of segmentation, and the aggregation method. The influence of these choices on the predictive performance was measured using a linear mixed model. In addition, models with generic features were trained and compared in terms of predictive performance and correlation.

RESULTS

No single choice consistently led to the best-performing models. In the mixed model, the choice of architecture (AUC + 0.016; p < 0.001), the level of feature extraction (AUC + 0.016; p < 0.001), and using all slices (AUC + 0.023; p < 0.001) were highly significant; using the segmentation had a lower influence (AUC + 0.011; p = 0.023), while the aggregation method was insignificant (p = 0.774). Models based on deep features were not significantly better than those based on generic features (p > 0.05 on all datasets). Deep feature sets correlated moderately with each other (r = 0.4), in contrast to generic feature sets (r = 0.89).

CONCLUSIONS

Different choices have a significant effect on the predictive performance of the resulting models; however, for the highest performance, these choices should be optimized during cross-validation.

Automatic Classification of Simulated Breast Tomosynthesis Whole Images for the Presence of Microcalcification Clusters Using Deep CNNs

Microcalcification clusters (MCs) are among the most important biomarkers for breast cancer, especially in cases of nonpalpable lesions. The vast majority of deep learning studies on digital breast tomosynthesis (DBT) are focused on detecting and classifying lesions, especially soft-tissue lesions, in small regions of interest previously selected. Only about 25% of the studies are specific to MCs, and all of them are based on the classification of small preselected regions. Classifying the whole image according to the presence or absence of MCs is a difficult task due to the size of MCs and all the information present in an entire image. A completely automatic and direct classification, which receives the entire image, without prior identification of any regions, is crucial for the usefulness of these techniques in a real clinical and screening environment. The main purpose of this work is to implement and evaluate the performance of convolutional neural networks (CNNs) regarding an automatic classification of a complete DBT image for the presence or absence of MCs (without any prior identification of regions). In this work, four popular deep CNNs are trained and compared with a new architecture proposed by us. The main task of these trainings was the classification of DBT cases by absence or presence of MCs. A public database of realistic simulated data was used, and the whole DBT image was taken into account as input. DBT data were considered without and with preprocessing (to study the impact of noise reduction and contrast enhancement methods on the evaluation of MCs with CNNs). The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance. Very promising results were achieved with a maximum AUC of 94.19% for the GoogLeNet. The second-best AUC value was obtained with a new implemented network, CNN-a, with 91.17%. This CNN had the particularity of also being the fastest, thus becoming a very interesting model to be considered in other studies. With this work, encouraging outcomes were achieved in this regard, obtaining similar results to other studies for the detection of larger lesions such as masses. Moreover, given the difficulty of visualizing the MCs, which are often spread over several slices, this work may have an important impact on the clinical analysis of DBT images.

Microcalcification Discrimination in Mammography Using Deep Convolutional Neural Network: Towards Rapid and Early Breast Cancer Diagnosis

Breast cancer is among the most common types of cancer in women and under the cases of misdiagnosed, or delayed in treatment, the mortality risk is high. The existence of breast microcalcifications is common in breast cancer patients and they are an effective indicator for early sign of breast cancer. However, microcalcifications are often missed and wrongly classified during screening due to their small sizes and indirect scattering in mammogram images. Motivated by this issue, this project proposes an adaptive transfer learning deep convolutional neural network in segmenting breast mammogram images with calcifications cases for early breast cancer diagnosis and intervention. Mammogram images of breast microcalcifications are utilized to train several deep neural network models and their performance is compared. Image filtering of the region of interest images was conducted to remove possible artifacts and noises to enhance the quality of the images before the training. Different hyperparameters such as epoch, batch size, etc were tuned to obtain the best possible result. In addition, the performance of the proposed fine-tuned hyperparameter of ResNet50 is compared with another state-of-the-art machine learning network such as ResNet34, VGG16, and AlexNet. Confusion matrices were utilized for comparison. The result from this study shows that the proposed ResNet50 achieves the highest accuracy with a value of 97.58%, followed by ResNet34 of 97.35%, VGG16 96.97%, and finally AlexNet of 83.06%.