A multi-task fusion model based on a residual-Multi-layer perceptron network for mammographic breast cancer screening

BACKGROUND AND OBJECTIVE

Deep learning approaches are being increasingly applied for medical computer-aided diagnosis (CAD). However, these methods generally target only specific image-processing tasks, such as lesion segmentation or benign state prediction. For the breast cancer screening task, single feature extraction models are generally used, which directly extract only those potential features from the input mammogram that are relevant to the target task. This can lead to the neglect of other important morphological features of the lesion as well as other auxiliary information from the internal breast tissue. To obtain more comprehensive and objective diagnostic results, in this study, we developed a multi-task fusion model that combines multiple specific tasks for CAD of mammograms.

METHODS

We first trained a set of separate, task-specific models, including a density classification model, a mass segmentation model, and a lesion benignity-malignancy classification model, and then developed a multi-task fusion model that incorporates all of the mammographic features from these different tasks to yield comprehensive and refined prediction results for breast cancer diagnosis.

RESULTS

The experimental results showed that our proposed multi-task fusion model outperformed other related state-of-the-art models in both breast cancer screening tasks in the publicly available datasets CBIS-DDSM and INbreast, achieving a competitive screening performance with area-under-the-curve scores of 0.92 and 0.95, respectively.

CONCLUSIONS

Our model not only allows an overall assessment of lesion types in mammography but also provides intermediate results related to radiological features and potential cancer risk factors, indicating its potential to offer comprehensive workflow support to radiologists.

Lesion mapping and functional characterization of hemiplegic children with different patterns of hand manipulation

Brain damage in children with unilateral cerebral palsy (UCP) affects motor function, with varying severity, making it difficult the performance of daily actions. Recently, qualitative and semi-quantitative methods have been developed for lesion classification, but studies on mild to moderate hand impairment are lacking. The present study aimed to characterize lesion topography and preserved brain areas in UCP children with specific patterns of hand manipulation. A homogeneous sample of 16 UCP children, aged 9 to 14 years, was enrolled in the study. Motor assessment included the characterization of the specific pattern of hand manipulation, by means of unimanual and bimanual measures (Kinematic Hand Classification, KHC; Manual Ability Classification System, MACS; House Functional Classification System, HFCS; Melbourne Unilateral Upper Limb Assessment, MUUL; Assisting Hand Assessment, AHA). The MRI morphological study included multiple methods: (a) qualitative lesion classification, (b) semi-quantitative classification (sq-MRI), (c) voxel-based morphometry comparing UCP and typically developed children (VBM-DARTEL), and (d) quantitative brain tissue segmentation (q-BTS). In addition, functional MRI was used to assess spared functional activations and cluster lateralization in the ipsilesional and contralesional hemispheres of UCP children during the execution of simple movements and grasping actions with the more affected hand. Lesions most frequently involved the periventricular white matter, corpus callosum, posterior limb of the internal capsule, thalamus, basal ganglia and brainstem. VMB-DARTEL analysis allowed to detect mainly white matter lesions. Both sq-MRI classification and q-BTS identified lesions of thalamus, brainstem, and basal ganglia. In particular, UCP patients with synergic hand pattern showed larger involvement of subcortical structures, as compared to those with semi-functional hand. Furthermore, sparing of gray matter in basal ganglia and thalamus was positively correlated with MUUL and AHA scores. Concerning white matter, q-BTS revealed a larger damage of fronto-striatal connections in patients with synergic hand, as compared to those with semi-functional hand. The volume of these connections was correlated to unimanual function (MUUL score). The fMRI results showed that all patients, but one, including those with cortical lesions, had activation in ipsilesional areas, regardless of lesion timing. Children with synergic hand showed more lateralized activation in the ipsilesional hemisphere both during grasping and simple movements, while children with semi-functional hand exhibited more bilateral activation during grasping. The study demonstrates that lesion localization, rather than lesion type based on the timing of their occurrence, is more associated with the functional level of hand manipulation. Overall, the preservation of subcortical structures and white matter can predict a better functional outcome. Future studies integrating different techniques (structural and functional imaging, TMS) could provide further evidence on the relation between brain reorganization and specific pattern of manipulation in UCP children.

Deep learning, radiomics and radiogenomics applications in the digital breast tomosynthesis: a systematic review

BACKGROUND

Recent advancements in computing power and state-of-the-art algorithms have helped in more accessible and accurate diagnosis of numerous diseases. In addition, the development of de novo areas in imaging science, such as radiomics and radiogenomics, have been adding more to personalize healthcare to stratify patients better. These techniques associate imaging phenotypes with the related disease genes. Various imaging modalities have been used for years to diagnose breast cancer. Nonetheless, digital breast tomosynthesis (DBT), a state-of-the-art technique, has produced promising results comparatively. DBT, a 3D mammography, is replacing conventional 2D mammography rapidly. This technological advancement is key to AI algorithms for accurately interpreting medical images.

OBJECTIVE AND METHODS

This paper presents a comprehensive review of deep learning (DL), radiomics and radiogenomics in breast image analysis. This review focuses on DBT, its extracted synthetic mammography (SM), and full-field digital mammography (FFDM). Furthermore, this survey provides systematic knowledge about DL, radiomics, and radiogenomics for beginners and advanced-level researchers.

RESULTS

A total of 500 articles were identified, with 30 studies included as the set criteria. Parallel benchmarking of radiomics, radiogenomics, and DL models applied to the DBT images could allow clinicians and researchers alike to have greater awareness as they consider clinical deployment or development of new models. This review provides a comprehensive guide to understanding the current state of early breast cancer detection using DBT images.

CONCLUSION

Using this survey, investigators with various backgrounds can easily seek interdisciplinary science and new DL, radiomics, and radiogenomics directions towards DBT.

Concatenated Xception-ResNet50 - A novel hybrid approach for accurate skin cancer prediction

Skin cancer is a malignant disease that affects millions of people around the world every year. It is an invasive disease characterised by an abnormal proliferation of skin cells in the body that multiply and spread through the lymph nodes, killing the surrounding tissue. The number of skin cancer cases is on the rise due to lifestyle changes and sun-seeking behaviour. As skin cancer is a deadly disease, early diagnosis and grading are crucial to save lives. In this work, state-of-the-art AI approaches are applied to develop a unique deep learning model that integrates Xception and ResNet50. This network achieves maximum accuracy by combining the properties of two robust networks. The proposed concatenated Xception-ResNet50 (X-R50) model can classify skin tumours as basal cell carcinoma, melanoma, melanocytic nevi, dermatofibroma, actinic keratoses and intraepithelial carcinoma, vascular and non-cancerous benign keratosis-like lesions. The performance of the proposed method is compared with a DeepCNN and other state-of-the-art transfer learning models. The Human Against Machine (HAM10000) dataset assesses the suggested method's performance. For this study, 10,500 skin images were used. The model is trained and tested with the sliding window technique. The proposed concatenated X-R50 model is cutting-edge, with a 97.8% prediction accuracy. The performance of the model is also validated by a statistical hypothesis test using analysis of variance (ANOVA). The reported approach is both accurate and efficient and can help dermatologists and clinicians detect skin cancer at an early stage of the clinical process.

The overview of the deep learning integrated into the medical imaging of liver: a review

Deep learning (DL) is a recently developed artificial intelligent method that can be integrated into numerous fields. For the imaging diagnosis of liver disease, several remarkable outcomes have been achieved with the application of DL currently. This advanced algorithm takes part in various sections of imaging processing such as liver segmentation, lesion delineation, disease classification, process optimization, etc. The DL optimized imaging diagnosis shows a broad prospect instead of the pathological biopsy for the advantages of convenience, safety, and inexpensiveness. In this paper, we reviewed the published representative DL-related hepatic imaging works, described the general situation of this new-rising technology in medical liver imaging and explored the future direction of DL development.

Diagnosis of skin diseases in the era of deep learning and mobile technology

Efficient methods developed with deep learning in the last ten years have provided objectivity and high accuracy in the diagnosis of skin diseases. They also support accurate, cost-effective and timely treatment. In addition, they provide diagnoses without the need to touch patients, which is very desirable when the disease is contagious or the patients have another contagious disease. On the other hand, it is not possible to run deep networks on resource-constrained devices (e.g., mobile phones). Therefore, lightweight network architectures have been proposed in the literature. However, merely a few mobile applications have been developed for the diagnosis of skin diseases from colored photographs using lightweight networks. Moreover, only a few types of skin diseases have been addressed in those applications. Additionally, they do not perform as well as the deep network models, particularly for pattern recognition. Therefore, in this study, a novel model has been constructed using MobileNet. Also, a novel loss function has been developed and used. The main contributions of this study are: (i) proposing a novel hybrid loss function; (ii) proposing a modified-MobileNet architecture; (iii) designing and implementing a mobile phone application with the modified-MobileNet and a user-friendly interface. Results indicated that the proposed technique can diagnose skin diseases with 94.76% accuracy.

CT window trainable neural network for improving intracranial hemorrhage detection by combining multiple settings

Window settings to rescale and contrast stretch raw data from radiographic images such as Computed Tomography (CT), X-ray and Magnetic Resonance images is a crucial step as data pre-processing to examine abnormalities and diagnose diseases. We propose a distant-supervised method for determining automatically the best window settings by attaching a window estimator module (WEM) to a deep convolutional neural network (DCNN)-based lesion classifier and training them in conjunction. Aside from predicting a flexible window setting for each raw image, we statistically identify the top four window settings by calculating the mean and standard deviations for the entire dataset. Images are scaled on each of the top settings estimated by WEM and following lesion classifiers are subsequently trained. We study the effects of only using the flexible window, the single fixed window as either a known default window used by radiologists or an estimated mean value, and two different approaches to combine results from the top window settings to improve the detection of intracranial hemorrhage (ICH) from brain CT images. Experimental results showed that using the top four window settings identified from the window estimator module and combining the results had the best performance.

Energy enhanced tissue texture in spectral computed tomography for lesion classification

Tissue texture reflects the spatial distribution of contrasts of image voxel gray levels, i.e., the tissue heterogeneity, and has been recognized as important biomarkers in various clinical tasks. Spectral computed tomography (CT) is believed to be able to enrich tissue texture by providing different voxel contrast images using different X-ray energies. Therefore, this paper aims to address two related issues for clinical usage of spectral CT, especially the photon counting CT (PCCT): (1) texture enhancement by spectral CT image reconstruction, and (2) spectral energy enriched tissue texture for improved lesion classification. For issue (1), we recently proposed a tissue-specific texture prior in addition to low rank prior for the individual energy-channel low-count image reconstruction problems in PCCT under the Bayesian theory. Reconstruction results showed the proposed method outperforms existing methods of total variation (TV), low-rank TV and tensor dictionary learning in terms of not only preserving texture features but also suppressing image noise. For issue (2), this paper will investigate three models to incorporate the enriched texture by PCCT in accordance with three types of inputs: one is the spectral images, another is the co-occurrence matrices (CMs) extracted from the spectral images, and the third one is the Haralick features (HF) extracted from the CMs. Studies were performed on simulated photon counting data by introducing attenuation-energy response curve to the traditional CT images from energy integration detectors. Classification results showed the spectral CT enriched texture model can improve the area under the receiver operating characteristic curve (AUC) score by 7.3%, 0.42% and 3.0% for the spectral images, CMs and HFs respectively on the five-energy spectral data over the original single energy data only. The CM- and HF-inputs can achieve the best AUC of 0.934 and 0.927. This texture themed study shows the insight that incorporating clinical important prior information, e.g., tissue texture in this paper, into the medical imaging, such as the upstream image reconstruction, the downstream diagnosis, and so on, can benefit the clinical tasks.

Texture Analysis Based on Auto-Mutual Information for Classifying Breast Lesions with Ultrasound

Described here is a novel texture extraction method based on auto-mutual information (AMI) for classifying breast lesions. The objective is to extract discriminating information found in the non-linear relationship of textures in breast ultrasound (BUS) images. The AMI method performs three basic tasks: (i) it transforms the input image using the ranklet transform to handle intensity variations of BUS images acquired with distinct ultrasound scanners; (ii) it extracts the AMI-based texture features in the horizontal and vertical directions from each ranklet image; and (iii) it classifies the breast lesions into benign and malignant classes, in which a support-vector machine is used as the underlying classifier. The image data set is composed of 2050 BUS images consisting of 1347 benign and 703 malignant tumors. Additionally, nine commonly used texture extraction methods proposed in the literature for BUS analysis are compared with the AMI method. The bootstrap method, which considers 1000 bootstrap samples, is used to evaluate classification performance. The experimental results indicate that the proposed approach outperforms its counterparts in terms of area under the receiver operating characteristic curve, sensitivity, specificity and Matthews correlation coefficient, with values of 0.82, 0.80, 0.85 and 0.63, respectively. These results suggest that the AMI method is suitable for breast lesion classification systems.

Classification of dental diseases using hyperspectral imaging and laser induced fluorescence

PURPOSE

Early diagnosis of tooth enamel demineralization, and dentin caries lesions, present a valuable solution to avoid or decrease their deleterious effect. The aim of this study was to design a simple, effective, and non-invasive technique, employing a novel laser wavelength to classify and differentiate between various tooth abnormalities in-vitro, by estimating wavelengths, showing distinctive appearance for each tooth class.

METHODS

This study implies a fluorescence hyperspectral imaging system employing a 395-nm laser diode source, irradiating a pre-diagnosed 12 molars and premolars teeth. The obtained reconstructed images were displayed and processed by HSAnalysis2XL, accompanied by a custom made digital, and image signal processing algorithms, revealing the exact wavelengths, characterizing the fluorescence of each tooth pre-diagnosed class.

RESULTS

The proposed hyperspectral imaging system was able to discriminate between normal, and abnormal dental classes for the entire specimens. Furthermore, a series of wavelengths, noting each lesion individually were obtained from the spectroscopic hyperspectral output. The root calculus, white spot, dentin caries, and enamel caries have a bright visual appearance at λ₃ = 702 nm, λ₅ = 771 nm, and λ₆ = 798 nm respectively. Consequently, these abnormalities exhibit a dark appearance at λ₁ = 421 nm, λ₂ = 462 nm, and λ₄ = 734 nm. The wavelength selections were confirmed by the grayscale image outcomes.

CONCLUSIONS

This study provides a set of wavelengths that can be employed by dentists to diagnose white spot, root calculus, and enamel dentin caries lesions under the irradiation of a new UV-vis laser illumination source without, any hazardous thermal or mechanical effects.