Automatic Segmentation of MR Brain Images With a Convolutional Neural Network

Automatic segmentation of white matter hyperintensities from brain magnetic resonance images in the era of deep learning and big data - A systematic review

BACKGROUND

White matter hyperintensities (WMH), of presumed vascular origin, are visible and quantifiable neuroradiological markers of brain parenchymal change. These changes may range from damage secondary to inflammation and other neurological conditions, through to healthy ageing. Fully automatic WMH quantification methods are promising, but still, traditional semi-automatic methods seem to be preferred in clinical research. We systematically reviewed the literature for fully automatic methods developed in the last five years, to assess what are considered state-of-the-art techniques, as well as trends in the analysis of WMH of presumed vascular origin.

METHOD

We registered the systematic review protocol with the International Prospective Register of Systematic Reviews (PROSPERO), registration number - CRD42019132200. We conducted the search for fully automatic methods developed from 2015 to July 2020 on Medline, Science direct, IEE Explore, and Web of Science. We assessed risk of bias and applicability of the studies using QUADAS 2.

RESULTS

The search yielded 2327 papers after removing 104 duplicates. After screening titles, abstracts and full text, 37 were selected for detailed analysis. Of these, 16 proposed a supervised segmentation method, 10 proposed an unsupervised segmentation method, and 11 proposed a deep learning segmentation method. Average DSC values ranged from 0.538 to 0.91, being the highest value obtained from an unsupervised segmentation method. Only four studies validated their method in longitudinal samples, and eight performed an additional validation using clinical parameters. Only 8/37 studies made available their methods in public repositories.

CONCLUSIONS

We found no evidence that favours deep learning methods over the more established k-NN, linear regression and unsupervised methods in this task. Data and code availability, bias in study design and ground truth generation influence the wider validation and applicability of these methods in clinical research.

Convolutional neural networks in medical image understanding: a survey

Imaging techniques are used to capture anomalies of the human body. The captured images must be understood for diagnosis, prognosis and treatment planning of the anomalies. Medical image understanding is generally performed by skilled medical professionals. However, the scarce availability of human experts and the fatigue and rough estimate procedures involved with them limit the effectiveness of image understanding performed by skilled medical professionals. Convolutional neural networks (CNNs) are effective tools for image understanding. They have outperformed human experts in many image understanding tasks. This article aims to provide a comprehensive survey of applications of CNNs in medical image understanding. The underlying objective is to motivate medical image understanding researchers to extensively apply CNNs in their research and diagnosis. A brief introduction to CNNs has been presented. A discussion on CNN and its various award-winning frameworks have been presented. The major medical image understanding tasks, namely image classification, segmentation, localization and detection have been introduced. Applications of CNN in medical image understanding of the ailments of brain, breast, lung and other organs have been surveyed critically and comprehensively. A critical discussion on some of the challenges is also presented.

Deep Learning Approach for Medical Image Analysis

Localization of region of interest (ROI) is paramount to the analysis of medical images to assist in the identification and detection of diseases. In this research, we explore the application of a deep learning approach in the analysis of some medical images. Traditional methods have been restricted due to the coarse and granulated appearance of most of these images. Recently, deep learning techniques have produced promising results in the segmentation of medical images for the diagnosis of diseases. This research experiments on medical images using a robust deep learning architecture based on the Fully Convolutional Network‐ (FCN‐) UNET method for the segmentation of three samples of medical images such as skin lesion, retinal images, and brain Magnetic Resonance Imaging (MRI) images. The proposed method can efficiently identify the ROI on these images to assist in the diagnosis of diseases such as skin cancer, eye defects and diabetes, and brain tumor. This system was evaluated on publicly available databases such as the International Symposium on Biomedical Imaging (ISBI) skin lesion images, retina images, and brain tumor datasets with over 90% accuracy and dice coefficient.

Deep Learning Approach for Generating MRA Images From 3D Quantitative Synthetic MRI Without Additional Scans

OBJECTIVES

Quantitative synthetic magnetic resonance imaging (MRI) enables synthesis of various contrast-weighted images as well as simultaneous quantification of T1 and T2 relaxation times and proton density. However, to date, it has been challenging to generate magnetic resonance angiography (MRA) images with synthetic MRI. The purpose of this study was to develop a deep learning algorithm to generate MRA images based on 3D synthetic MRI raw data.

MATERIALS AND METHODS

Eleven healthy volunteers and 4 patients with intracranial aneurysms were included in this study. All participants underwent a time-of-flight (TOF) MRA sequence and a 3D-QALAS synthetic MRI sequence. The 3D-QALAS sequence acquires 5 raw images, which were used as the input for a deep learning network. The input was converted to its corresponding MRA images by a combination of a single-convolution and a U-net model with a 5-fold cross-validation, which were then compared with a simple linear combination model. Image quality was evaluated by calculating the peak signal-to-noise ratio (PSNR), structural similarity index measurements (SSIMs), and high frequency error norm (HFEN). These calculations were performed for deep learning MRA (DL-MRA) and linear combination MRA (linear-MR), relative to TOF-MRA, and compared with each other using a nonparametric Wilcoxon signed-rank test. Overall image quality and branch visualization, each scored on a 5-point Likert scale, were blindly and independently rated by 2 board-certified radiologists.

RESULTS

Deep learning MRA was successfully obtained in all subjects. The mean PSNR, SSIM, and HFEN of the DL-MRA were significantly higher, higher, and lower, respectively, than those of the linear-MRA (PSNR, 35.3 ± 0.5 vs 34.0 ± 0.5, P < 0.001; SSIM, 0.93 ± 0.02 vs 0.82 ± 0.02, P < 0.001; HFEN, 0.61 ± 0.08 vs 0.86 ± 0.05, P < 0.001). The overall image quality of the DL-MRA was comparable to that of TOF-MRA (4.2 ± 0.7 vs 4.4 ± 0.7, P = 0.99), and both types of images were superior to that of linear-MRA (1.5 ± 0.6, for both P < 0.001). No significant differences were identified between DL-MRA and TOF-MRA in the branch visibility of intracranial arteries, except for ophthalmic artery (1.2 ± 0.5 vs 2.3 ± 1.2, P < 0.001).

CONCLUSIONS

Magnetic resonance angiography generated by deep learning from 3D synthetic MRI data visualized major intracranial arteries as effectively as TOF-MRA, with inherently aligned quantitative maps and multiple contrast-weighted images. Our proposed algorithm may be useful as a screening tool for intracranial aneurysms without requiring additional scanning time.

Deep learning for brain disorders: from data processing to disease treatment

In order to reach precision medicine and improve patients' quality of life, machine learning is increasingly used in medicine. Brain disorders are often complex and heterogeneous, and several modalities such as demographic, clinical, imaging, genetics and environmental data have been studied to improve their understanding. Deep learning, a subpart of machine learning, provides complex algorithms that can learn from such various data. It has become state of the art in numerous fields, including computer vision and natural language processing, and is also growingly applied in medicine. In this article, we review the use of deep learning for brain disorders. More specifically, we identify the main applications, the concerned disorders and the types of architectures and data used. Finally, we provide guidelines to bridge the gap between research studies and clinical routine.

Detection Methods of COVID-19

Since being first detected in China, coronavirus disease 2019 (COVID-19) has spread rapidly across the world, triggering a global pandemic with no viable cure in sight. As a result, national responses have focused on the effective minimization of the spread. Border control measures and travel restrictions have been implemented in a number of countries to limit the import and export of the virus. The detection of COVID-19 is a key task for physicians. The erroneous results of early laboratory tests and their delays led researchers to focus on different options. Information obtained from computed tomography (CT) and radiological images is important for clinical diagnosis. Therefore, it is worth developing a rapid method of detection of viral diseases through the analysis of radiographic images. We propose a novel method of detection of COVID-19. The purpose is to provide clinical decision support to healthcare workers and researchers. The article is to support researchers working on early detection of COVID-19 as well as similar viral diseases.

Accurate deep neural network model to detect cardiac arrhythmia on more than 10,000 individual subject ECG records

BACKGROUND AND OBJECTIVE

Cardiac arrhythmia, which is an abnormal heart rhythm, is a common clinical problem in cardiology. Detection of arrhythmia on an extended duration electrocardiogram (ECG) is done based on initial algorithmic software screening, with final visual validation by cardiologists. It is a time consuming and subjective process. Therefore, fully automated computer-assisted detection systems with a high degree of accuracy have an essential role in this task. In this study, we proposed an effective deep neural network (DNN) model to detect different rhythm classes from a new ECG database.

METHODS

Our DNN model was designed for high performance on all ECG leads. The proposed model, which included both representation learning and sequence learning tasks, showed promising results on all 12-lead inputs. Convolutional layers and sub-sampling layers were used in the representation learning phase. The sequence learning part involved a long short-term memory (LSTM) unit after representation of learning layers.

RESULTS

We performed two different class scenarios, including reduced rhythms (seven rhythm types) and merged rhythms (four rhythm types) according to the records from the database. Our trained DNN model achieved 92.24% and 96.13% accuracies for the reduced and merged rhythm classes, respectively.

CONCLUSION

Recently, deep learning algorithms have been found to be useful because of their high performance. The main challenge is the scarcity of appropriate training and testing resources because model performance is dependent on the quality and quantity of case samples. In this study, we used a new public arrhythmia database comprising more than 10,000 records. We constructed an efficient DNN model for automated detection of arrhythmia using these records.

Deep Neural Network-Based Sinogram Super-Resolution and Bandwidth Enhancement for Limited-Data Photoacoustic Tomography

Photoacoustic tomography (PAT) is a noninvasive imaging modality combining the benefits of optical contrast at ultrasonic resolution. Analytical reconstruction algorithms for photoacoustic (PA) signals require a large number of data points for accurate image reconstruction. However, in practical scenarios, data are collected using the limited number of transducers along with data being often corrupted with noise resulting in only qualitative images. Furthermore, the collected boundary data are band-limited due to limited bandwidth (BW) of the transducer, making the PA imaging with limited data being qualitative. In this work, a deep neural network-based model with loss function being scaled root-mean-squared error was proposed for super-resolution, denoising, as well as BW enhancement of the PA signals collected at the boundary of the domain. The proposed network has been compared with traditional as well as other popular deep-learning methods in numerical as well as experimental cases and is shown to improve the collected boundary data, in turn, providing superior quality reconstructed PA image. The improvement obtained in the Pearson correlation, structural similarity index metric, and root-mean-square error was as high as 35.62%, 33.81%, and 41.07%, respectively, for phantom cases and signal-to-noise ratio improvement in the reconstructed PA images was as high as 11.65 dB for in vivo cases compared with reconstructed image obtained using original limited BW data. Code is available at https://sites.google.com/site/sercmig/home/dnnpat.

A Three-Dimensional Deep Convolutional Neural Network for Automatic Segmentation and Diameter Measurement of Type B Aortic Dissection

OBJECTIVE

To provide an automatic method for segmentation and diameter measurement of type B aortic dissection (TBAD).

MATERIALS AND METHODS

Aortic computed tomography angiographic images from 139 patients with TBAD were consecutively collected. We implemented a deep learning method based on a three-dimensional (3D) deep convolutional neural (CNN) network, which realizes automatic segmentation and measurement of the entire aorta (EA), true lumen (TL), and false lumen (FL). The accuracy, stability, and measurement time were compared between deep learning and manual methods. The intra- and inter-observer reproducibility of the manual method was also evaluated.

RESULTS

The mean dice coefficient scores were 0.958, 0.961, and 0.932 for EA, TL, and FL, respectively. There was a linear relationship between the reference standard and measurement by the manual and deep learning method (r = 0.964 and 0.991, respectively). The average measurement error of the deep learning method was less than that of the manual method (EA, 1.64% vs. 4.13%; TL, 2.46% vs. 11.67%; FL, 2.50% vs. 8.02%). Bland-Altman plots revealed that the deviations of the diameters between the deep learning method and the reference standard were -0.042 mm (-3.412 to 3.330 mm), -0.376 mm (-3.328 to 2.577 mm), and 0.026 mm (-3.040 to 3.092 mm) for EA, TL, and FL, respectively. For the manual method, the corresponding deviations were -0.166 mm (-1.419 to 1.086 mm), -0.050 mm (-0.970 to 1.070 mm), and -0.085 mm (-1.010 to 0.084 mm). Intra- and inter-observer differences were found in measurements with the manual method, but not with the deep learning method. The measurement time with the deep learning method was markedly shorter than with the manual method (21.7 ± 1.1 vs. 82.5 ± 16.1 minutes, p < 0.001).

CONCLUSION

The performance of efficient segmentation and diameter measurement of TBADs based on the 3D deep CNN was both accurate and stable. This method is promising for evaluating aortic morphology automatically and alleviating the workload of radiologists in the near future.

Blade Rub-Impact Fault Identification Using Autoencoder-Based Nonlinear Function Approximation and a Deep Neural Network

A blade rub-impact fault is one of the complex and frequently appearing faults in turbines. Due to their nonlinear and nonstationary nature, complex signal analysis techniques, which are expensive in terms of computation time, are required to extract valuable fault information from the vibration signals collected from rotor systems. In this work, a novel method for diagnosing the blade rub-impact faults of different severity levels is proposed. Specifically, the deep undercomplete denoising autoencoder is first used for estimating the nonlinear function of the system under normal operating conditions. Next, the residual signals obtained as the difference between the original signals and their estimates by the autoencoder are computed. Finally, these residual signals are used as inputs to a deep neural network to determine the current state of the rotor system. The experimental results demonstrate that the amplitudes of the residual signals reflect the changes in states of the rotor system and the fault severity levels. Furthermore, these residual signals in combination with the deep neural network demonstrated promising fault identification results when applied to a complex nonlinear fault, such as a blade-rubbing fault. To test the effectiveness of the proposed nonlinear-based fault diagnosis algorithm, this technique is compared with the autoregressive with external input Laguerre proportional-integral observer that is a linear-based fault diagnosis observation technique.

Brain volumes quantification from MRI in healthy controls: Assessing correlation, agreement and robustness of a convolutional neural network-based software against FreeSurfer, CAT12 and FSL

BACKGROUND AND PURPOSE

There are instances in which an estimate of the brain volume should be obtained from MRI in clinical practice. Our objective is to calculate cross-sectional robustness of a convolutional neural network (CNN) based software (Entelai Pic) for brain volume estimation and compare it to traditional software such as FreeSurfer, CAT12 and FSL in healthy controls (HC).

MATERIALS AND METHODS

Sixteen HC were scanned four times, two different days on two different MRI scanners (1.5 T and 3 T). Volumetric T1-weighted images were acquired and post-processed with FreeSurfer v6.0.0, Entelai Pic v2, CAT12 v12.5 and FSL v5.0.9. Whole-brain, grey matter (GM), white matter (WM) and cerebrospinal fluid (CSF) volumes were calculated. Correlation and agreement between methods was assessed using intraclass correlation coefficient (ICC) and Bland Altman plots. Robustness was assessed using the coefficient of variation (CV).

RESULTS

Whole-brain volume estimation had better correlation between FreeSurfer and Entelai Pic (ICC (95% CI) 0.96 (0.94-0.97)) than FreeSurfer and CAT12 (0.92 (0.88-0.96)) and FSL (0.87 (0.79-0.91)). WM, GM and CSF showed a similar trend. Compared to FreeSurfer, Entelai Pic provided similarly robust segmentations of brain volumes both on same-scanner (mean CV 1.07, range 0.20-3.13% vs. mean CV 1.05, range 0.21-3.20%, p = 0.86) and on different-scanner variables (mean CV 3.84, range 2.49-5.91% vs. mean CV 3.84, range 2.62-5.13%, p = 0.96). Mean post-processing times were 480, 5, 40 and 5 min for FreeSurfer, Entelai Pic, CAT12 and FSL respectively.

CONCLUSION

Based on robustness and processing times, our CNN-based model is suitable for cross-sectional volumetry on clinical practice.

Fully Automated and Standardized Segmentation of Adipose Tissue Compartments via Deep Learning in 3D Whole-Body MRI of Epidemiologic Cohort Studies

PURPOSE

To enable fast and reliable assessment of subcutaneous and visceral adipose tissue compartments derived from whole-body MRI.

MATERIALS AND METHODS

Quantification and localization of different adipose tissue compartments derived from whole-body MR images is of high interest in research concerning metabolic conditions. For correct identification and phenotyping of individuals at increased risk for metabolic diseases, a reliable automated segmentation of adipose tissue into subcutaneous and visceral adipose tissue is required. In this work, a three-dimensional (3D) densely connected convolutional neural network (DCNet) is proposed to provide robust and objective segmentation. In this retrospective study, 1000 cases (average age, 66 years ± 13 [standard deviation]; 523 women) from the Tuebingen Family Study database and the German Center for Diabetes research database and 300 cases (average age, 53 years ± 11; 152 women) from the German National Cohort (NAKO) database were collected for model training, validation, and testing, with transfer learning between the cohorts. These datasets included variable imaging sequences, imaging contrasts, receiver coil arrangements, scanners, and imaging field strengths. The proposed DCNet was compared to a similar 3D U-Net segmentation in terms of sensitivity, specificity, precision, accuracy, and Dice overlap.

RESULTS

Fast (range, 5-7 seconds) and reliable adipose tissue segmentation can be performed with high Dice overlap (0.94), sensitivity (96.6%), specificity (95.1%), precision (92.1%), and accuracy (98.4%) from 3D whole-body MRI datasets (field of view coverage, 450 × 450 × 2000 mm). Segmentation masks and adipose tissue profiles are automatically reported back to the referring physician.

CONCLUSION

Automated adipose tissue segmentation is feasible in 3D whole-body MRI datasets and is generalizable to different epidemiologic cohort studies with the proposed DCNet.Supplemental material is available for this article.© RSNA, 2020.

Findings from machine learning in clinical medical imaging applications - Lessons for translation to the forensic setting

Machine learning (ML) techniques are increasingly being used in clinical medical imaging to automate distinct processing tasks. In post-mortem forensic radiology, the use of these algorithms presents significant challenges due to variability in organ position, structural changes from decomposition, inconsistent body placement in the scanner, and the presence of foreign bodies. Existing ML approaches in clinical imaging can likely be transferred to the forensic setting with careful consideration to account for the increased variability and temporal factors that affect the data used to train these algorithms. Additional steps are required to deal with these issues, by incorporating the possible variability into the training data through data augmentation, or by using atlases as a pre-processing step to account for death-related factors. A key application of ML would be then to highlight anatomical and gross pathological features of interest, or present information to help optimally determine the cause of death. In this review, we highlight results and limitations of applications in clinical medical imaging that use ML to determine key implications for their application in the forensic setting.

Randomized Controlled Early versus Late Ventricular Intervention Study in Posthemorrhagic Ventricular Dilatation: Outcome at 2 Years

OBJECTIVE

To compare the effect of intervention at low vs high threshold of ventriculomegaly in preterm infants with posthemorrhagic ventricular dilatation on death or severe neurodevelopmental disability.

STUDY DESIGN

This multicenter randomized controlled trial reviewed lumbar punctures initiated after either a low threshold (ventricular index of >p97 and anterior horn width of >6 mm) or high threshold (ventricular index of >p97 + 4 mm and anterior horn width of >10 mm). The composite adverse outcome was defined as death or cerebral palsy or Bayley composite cognitive/motor scores <-2 SDs at 24 months corrected age.

RESULTS

Outcomes were assessed in 113 of 126 infants. The composite adverse outcome was seen in 20 of 58 infants (35%) in the low threshold group and 28 of 55 (51%) in the high threshold (P = .07). The low threshold intervention was associated with a decreased risk of an adverse outcome after correcting for gestational age, severity of intraventricular hemorrhage, and cerebellar hemorrhage (aOR, 0.24; 95% CI, 0.07-0.87; P = .03). Infants with a favorable outcome had a smaller fronto-occipital horn ratio (crude mean difference, -0.06; 95% CI, -0.09 to -0.03; P < .001) at term-equivalent age. Infants in the low threshold group with a ventriculoperitoneal shunt, had cognitive and motor scores similar to those without (P = .3 for both), whereas in the high threshold group those with a ventriculoperitoneal shunt had significantly lower scores than those without a ventriculoperitoneal shunt (P = .01 and P = .004, respectively).

CONCLUSIONS

In a post hoc analysis, earlier intervention was associated with a lower odds of death or severe neurodevelopmental disability in preterm infants with progressive posthemorrhagic ventricular dilatation.

TRIAL REGISTRATION

ISRCTN43171322.

IA-net: informative attention convolutional neural network for choroidal neovascularization segmentation in OCT images

Choroidal neovascularization (CNV) is a characteristic feature of wet age-related macular degeneration (AMD). Quantification of CNV is useful to clinicians in the diagnosis and treatment of CNV disease. Before quantification, CNV lesion should be delineated by automatic CNV segmentation technology. Recently, deep learning methods have achieved significant success for medical image segmentation. However, some CNVs are small objects which are hard to discriminate, resulting in performance degradation. In addition, it's difficult to train an effective network for accurate segmentation due to the complicated characteristics of CNV in OCT images. In order to tackle these two challenges, this paper proposed a novel Informative Attention Convolutional Neural Network (IA-net) for automatic CNV segmentation in OCT images. Considering that the attention mechanism has the ability to enhance the discriminative power of the interesting regions in the feature maps, the attention enhancement block is developed by introducing the additional attention constraint. It has the ability to force the model to pay high attention on CNV in the learned feature maps, improving the discriminative ability of the learned CNV features, which is useful to improve the segmentation performance on small CNV. For accurate pixel classification, the novel informative loss is proposed with the incorporation of an informative attention map. It can focus training on a set of informative samples that are difficult to be predicted. Therefore, the trained model has the ability to learn enough information to classify these informative samples, further improving the performance. The experimental results on our database demonstrate that the proposed method outperforms traditional CNV segmentation methods.

Gray Matter Segmentation of Brain MRI Using Hybrid Enhanced Independent Component Analysis in Noisy and Noise Free Environment

Medical segmentation is the primary task performed to diagnosis the abnormalities in the human body. The brain is the complex organ and anatomical segmentation of brain tissues is a challenging task. In this paper, we used Enhanced Independent component analysis to perform the segmentation of gray matter. We used modified K means, Expected Maximization and Hidden Markov random field to provide better spatial correlation that overcomes in-homogeneity, noise and low contrast. Our objective is achieved in two steps (i) initially unwanted tissues are clipped from the MRI image using skull stripped Algorithm (ii) Enhanced Independent Component analysis is used to perform the segmentation of gray matter. We apply the proposed method on both T1w and T2w MRI to perform segmentation of gray matter at different noisy environments. We evaluate the the performance of our proposed system with Jaccard Index, Dice Coefficient and Accuracy. We further compared the proposed system performance with the existing frameworks. Our proposed method gives better segmentation of gray matter useful for diagnosis neurodegenerative disorders.

Deep Learning: A Breakthrough in Medical Imaging

Deep learning has attracted great attention in the medical imaging community as a promising solution for automated, fast and accurate medical image analysis, which is mandatory for quality healthcare. Convolutional neural networks and its variants have become the most preferred and widely used deep learning models in medical image analysis. In this paper, concise overviews of the modern deep learning models applied in medical image analysis are provided and the key tasks performed by deep learning models, i.e. classification, segmentation, retrieval, detection, and registration are reviewed in detail. Some recent researches have shown that deep learning models can outperform medical experts in certain tasks. With the significant breakthroughs made by deep learning methods, it is expected that patients will soon be able to safely and conveniently interact with AI-based medical systems and such intelligent systems will actually improve patient healthcare. There are various complexities and challenges involved in deep learning-based medical image analysis, such as limited datasets. But researchers are actively working in this area to mitigate these challenges and further improve health care with AI.

OpSeF: Open Source Python Framework for Collaborative Instance Segmentation of Bioimages

Various pre-trained deep learning models for the segmentation of bioimages have been made available as developer-to-end-user solutions. They are optimized for ease of use and usually require neither knowledge of machine learning nor coding skills. However, individually testing these tools is tedious and success is uncertain. Here, we present the Open Segmentation Framework (OpSeF), a Python framework for deep learning-based instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts' knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. OpSeF semi-automates preprocessing, convolutional neural network (CNN)-based segmentation in 2D or 3D, and postprocessing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for pre- and postprocessing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNN-architecture in which the biomedical user might invest the effort of manually labeling training data. We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows. Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little; the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.

FastSurfer - A fast and accurate deep learning based neuroimaging pipeline

Traditional neuroimage analysis pipelines involve computationally intensive, time-consuming optimization steps, and thus, do not scale well to large cohort studies with thousands or tens of thousands of individuals. In this work we propose a fast and accurate deep learning based neuroimaging pipeline for the automated processing of structural human brain MRI scans, replicating FreeSurfer's anatomical segmentation including surface reconstruction and cortical parcellation. To this end, we introduce an advanced deep learning architecture capable of whole-brain segmentation into 95 classes. The network architecture incorporates local and global competition via competitive dense blocks and competitive skip pathways, as well as multi-slice information aggregation that specifically tailor network performance towards accurate segmentation of both cortical and subcortical structures. Further, we perform fast cortical surface reconstruction and thickness analysis by introducing a spectral spherical embedding and by directly mapping the cortical labels from the image to the surface. This approach provides a full FreeSurfer alternative for volumetric analysis (in under 1 ​min) and surface-based thickness analysis (within only around 1 ​h runtime). For sustainability of this approach we perform extensive validation: we assert high segmentation accuracy on several unseen datasets, measure generalizability and demonstrate increased test-retest reliability, and high sensitivity to group differences in dementia.

Prognostic Value and Reproducibility of AI-assisted Analysis of Lung Involvement in COVID-19 on Low-Dose Submillisievert Chest CT: Sample Size Implications for Clinical Trials

PURPOSE

To compare the prognostic value and reproducibility of visual versus AI-assisted analysis of lung involvement on submillisievert low-dose chest CT in COVID-19 patients.

MATERIALS AND METHODS

This was a HIPAA-compliant, institutional review board-approved retrospective study. From March 15 to June 1, 2020, 250 RT-PCR confirmed COVID-19 patients were studied with low-dose chest CT at admission. Visual and AI-assisted analysis of lung involvement was performed by using a semi-quantitative CT score and a quantitative percentage of lung involvement. Adverse outcome was defined as intensive care unit (ICU) admission or death. Cox regression analysis, Kaplan-Meier curves, and cross-validated receiver operating characteristic curve with area under the curve (AUROC) analysis was performed to compare model performance. Intraclass correlation coefficients (ICCs) and Bland- Altman analysis was used to assess intra- and interreader reproducibility.

RESULTS

Adverse outcome occurred in 39 patients (11 deaths, 28 ICU admissions). AUC values from AI-assisted analysis were significantly higher than those from visual analysis for both semi-quantitative CT scores and percentages of lung involvement (all P<0.001). Intrareader and interreader agreement rates were significantly higher for AI-assisted analysis than visual analysis (all ICC ≥0.960 versus ≥0.885). AI-assisted variability for quantitative percentage of lung involvement was 17.2% (coefficient of variation) versus 34.7% for visual analysis. The sample size to detect a 5% change in lung involvement with 90% power and an α error of 0.05 was 250 patients with AI-assisted analysis and 1014 patients with visual analysis.

CONCLUSION

AI-assisted analysis of lung involvement on submillisievert low-dose chest CT outperformed conventional visual analysis in predicting outcome in COVID-19 patients while reducing CT variability. Lung involvement on chest CT could be used as a reliable metric in future clinical trials.

Deep Learning Signature Based on Staging CT for Preoperative Prediction of Sentinel Lymph Node Metastasis in Breast Cancer

RATIONALE AND OBJECTIVES

To evaluate the noninvasive predictive performance of deep learning features based on staging CT for sentinel lymph node (SLN) metastasis of breast cancer.

MATERIALS AND METHODS

A total of 348 breast cancer patients were enrolled in this study, with their SLN metastases pathologically confirmed. All patients received contrast-enhanced CT preoperative examinations and CT images were segmented and analyzed to extract deep features. After the feature selection, deep learning signature was built with the selected key features. The performance of the deep learning signatures was assessed with respect to discrimination, calibration, and clinical usefulness in the primary cohort (184 patients from January 2016 to March 2017) and then validated in the independent validation cohort (164 patients from April 2017 to December 2018).

RESULTS

Ten deep learning features were automatically selected in the primary cohort to establish the deep learning signature of SLN metastasis. The deep learning signature shows favorable discriminative ability with an area under curve of 0.801 (95% confidence interval: 0.736-0.867) in primary cohort and 0.817 (95% confidence interval: 0.751-0.884) in validation cohort. To further distinguish the number of metastatic SLNs (1-2 or more than two metastatic SLN), another deep learning signature was constructed and also showed moderate performance (area under curve 0.770).

CONCLUSION

We developed the deep learning signatures for preoperative prediction of SLN metastasis status and numbers (1-2 or more than two metastatic SLN) in patients with breast cancer. The deep learning signature may potentially provide a noninvasive approach to assist clinicians in predicting SLN metastasis in patients with breast cancer.

Head Circumference Growth Is Enhanced by SMOFlipid in Preterm Neonates

BACKGROUND

Suboptimal fat intake during the early postnatal weeks significantly affects brain growth and maturation. Studies to date have focused on the quantity rather than the quality of fat intake.

OBJECTIVE

We hypothesized that early nutrition of premature neonates should also include optimization of the type of fat intake, and thus those receiving SMOFlipid, a balanced multicomponent lipid emulsion, would have improved head growth as measured by head circumference (HC) at discharge.

STUDY DESIGN

We retrospectively reviewed HC in infants weighing <1,500 g who were hospitalized for two or more weeks during a 20-month period, in which all preterm infants received fat as Lipofundin, and the following 20-month period, in which all such infants received SMOFlipid.Lipids were dosed up to 3 g/kg/day and reduced as enteral nutrition progressed. Parenteral fish oil (Omegaven) was permitted as rescue therapy during both periods.

RESULTS

Period 2 infants had better head growth (0.79 [0.69,0.90] vs. 0.75 [0.64,0.86] cm/week; p = 0.0158). More infants reached discharge with an HC of ≥50 percentile (51 vs. 31%; p = 0.0007), and fewer infants had an HC of ≤3 percentile (11 vs. 14%; p = 0.023). Median length of stay was reduced by more than 1 week.A multivariable regression was performed using the weekly increase in HC as the dependent variable, and the time epoch, birth weight, gestational age, hospitalization days, and gender as independent variables. Only the time epoch and days of hospitalization were significant (both p < 0.0001).

CONCLUSION

Our data offer preliminary evidence of improved brain growth in those receiving a balanced lipid emulsion as compared with a soybean oil emulsion.

Bearing Fault Identification Using Machine Learning and Adaptive Cascade Fault Observer

In this work, a hybrid procedure for bearing fault identification using a machine learning and adaptive cascade observer is explained. To design an adaptive cascade observer, the normal signal approximation is the first step. Therefore, the fuzzy orthonormal regressive (FOR) technique was developed to approximate the acoustic emission (AE) and vibration (non-stationary and nonlinear) bearing signals in normal conditions. After approximating the normal signal of bearing using the FOR technique, the adaptive cascade observer is modeled in four steps. First, the linear observation technique using a FOR proportional-integral (PI) observer (FOR-PIO) is developed. In the second step, to increase the power of uncertaintie rejection (robustness) of the FOR-PIO, the structure procedure is used serially. Next, the fuzzy like observer is selected to increase the accuracy of FOR structure PI observer (FOR-SPIO). Moreover, the adaptive technique is used to develop the reliability of the cascade (fuzzy-structure PI) observer. Additionally to fault identification, the machine-learning algorithm using a support vector machine (SVM) is recommended. The effectiveness of the adaptive cascade observer with the SVM fault identifier was validated by a vibration and AE datasets. Based on the results, the average vibration and AE fault diagnosis using the adaptive cascade observer with the SVM fault identifier are 97.8% and 97.65%, respectively.

Automated thalamic nuclei segmentation using multi-planar cascaded convolutional neural networks

PURPOSE

To develop a fast and accurate convolutional neural network based method for segmentation of thalamic nuclei.

METHODS

A cascaded multi-planar scheme with a modified residual U-Net architecture was used to segment thalamic nuclei on conventional and white-matter-nulled (WMn) magnetization prepared rapid gradient echo (MPRAGE) data. A single network was optimized to work with images from healthy controls and patients with multiple sclerosis (MS) and essential tremor (ET), acquired at both 3 T and 7 T field strengths. WMn-MPRAGE images were manually delineated by a trained neuroradiologist using the Morel histological atlas as a guide to generate reference ground truth labels. Dice similarity coefficient and volume similarity index (VSI) were used to evaluate performance. Clinical utility was demonstrated by applying this method to study the effect of MS on thalamic nuclei atrophy.

RESULTS

Segmentation of each thalamus into twelve nuclei was achieved in under a minute. For 7 T WMn-MPRAGE, the proposed method outperforms current state-of-the-art on patients with ET with statistically significant improvements in Dice for five nuclei (increase in the range of 0.05-0.18) and VSI for four nuclei (increase in the range of 0.05-0.19), while performing comparably for healthy and MS subjects. Dice and VSI achieved using 7 T WMn-MPRAGE data are comparable to those using 3 T WMn-MPRAGE data. For conventional MPRAGE, the proposed method shows a statistically significant Dice improvement in the range of 0.14-0.63 over FreeSurfer for all nuclei and disease types. Effect of noise on network performance shows robustness to images with SNR as low as half the baseline SNR. Atrophy of four thalamic nuclei and whole thalamus was observed for MS patients compared to healthy control subjects, after controlling for the effect of parallel imaging, intracranial volume, gender, and age (p < 0.004).

CONCLUSION

The proposed segmentation method is fast, accurate, performs well across disease types and field strengths, and shows great potential for improving our understanding of thalamic nuclei involvement in neurological diseases.

Variability and reproducibility in deep learning for medical image segmentation

Medical image segmentation is an important tool for current clinical applications. It is the backbone of numerous clinical diagnosis methods, oncological treatments and computer-integrated surgeries. A new class of machine learning algorithm, deep learning algorithms, outperforms the results of classical segmentation in terms of accuracy. However, these techniques are complex and can have a high range of variability, calling the reproducibility of the results into question. In this article, through a literature review, we propose an original overview of the sources of variability to better understand the challenges and issues of reproducibility related to deep learning for medical image segmentation. Finally, we propose 3 main recommendations to address these potential issues: (1) an adequate description of the framework of deep learning, (2) a suitable analysis of the different sources of variability in the framework of deep learning, and (3) an efficient system for evaluating the segmentation results.

Automatic segmentation of brain MRI using a novel patch-wise U-net deep architecture

Accurate segmentation of brain magnetic resonance imaging (MRI) is an essential step in quantifying the changes in brain structure. Deep learning in recent years has been extensively used for brain image segmentation with highly promising performance. In particular, the U-net architecture has been widely used for segmentation in various biomedical related fields. In this paper, we propose a patch-wise U-net architecture for the automatic segmentation of brain structures in structural MRI. In the proposed brain segmentation method, the non-overlapping patch-wise U-net is used to overcome the drawbacks of conventional U-net with more retention of local information. In our proposed method, the slices from an MRI scan are divided into non-overlapping patches that are fed into the U-net model along with their corresponding patches of ground truth so as to train the network. The experimental results show that the proposed patch-wise U-net model achieves a Dice similarity coefficient (DSC) score of 0.93 in average and outperforms the conventional U-net and the SegNet-based methods by 3% and 10%, respectively, for on Open Access Series of Imaging Studies (OASIS) and Internet Brain Segmentation Repository (IBSR) dataset.

Multi-view secondary input collaborative deep learning for lung nodule 3D segmentation

BACKGROUND

Convolutional neural networks (CNNs) have been extensively applied to two-dimensional (2D) medical image segmentation, yielding excellent performance. However, their application to three-dimensional (3D) nodule segmentation remains a challenge.

METHODS

In this study, we propose a multi-view secondary input residual (MV-SIR) convolutional neural network model for 3D lung nodule segmentation using the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset of chest computed tomography (CT) images. Lung nodule cubes are prepared from the sample CT images. Further, from the axial, coronal, and sagittal perspectives, multi-view patches are generated with randomly selected voxels in the lung nodule cubes as centers. Our model consists of six submodels, which enable learning of 3D lung nodules sliced into three views of features; each submodel extracts voxel heterogeneity and shape heterogeneity features. We convert the segmentation of 3D lung nodules into voxel classification by inputting the multi-view patches into the model and determine whether the voxel points belong to the nodule. The structure of the secondary input residual submodel comprises a residual block followed by a secondary input module. We integrate the six submodels to classify whether voxel points belong to nodules, and then reconstruct the segmentation image.

RESULTS

The results of tests conducted using our model and comparison with other existing CNN models indicate that the MV-SIR model achieves excellent results in the 3D segmentation of pulmonary nodules, with a Dice coefficient of 0.926 and an average surface distance of 0.072.

CONCLUSION

our MV-SIR model can accurately perform 3D segmentation of lung nodules with the same segmentation accuracy as the U-net model.

Developing intelligent medical image modality classification system using deep transfer learning and LDA

Rapid advancement in imaging technology generates an enormous amount of heterogeneous medical data for disease diagnosis and rehabilitation process. Radiologists may require related clinical cases from medical archives for analysis and disease diagnosis. It is challenging to retrieve the associated clinical cases automatically, efficiently and accurately from the substantial medical image archive due to diversity in diseases and imaging modalities. We proposed an efficient and accurate approach for medical image modality classification that can used for retrieval of clinical cases from large medical repositories. The proposed approach is developed using transfer learning concept with pre-trained ResNet50 Deep learning model for optimized features extraction followed by linear discriminant analysis classification (TLRN-LDA). Extensive experiments are performed on challenging standard benchmark ImageCLEF-2012 dataset of 31 classes. The developed approach yields improved average classification accuracy of 87.91%, which is higher up-to 10% compared to the state-of-the-art approaches on the same dataset. Moreover, hand-crafted features are extracted for comparison. Performance of TLRN-LDA system demonstrates the effectiveness over state-of-the-art systems. The developed approach may be deployed to diagnostic centers to assist the practitioners for accurate and efficient clinical case retrieval and disease diagnosis.

Artificial intelligence in radiotherapy: a technological review

Radiation therapy (RT) is widely used to treat cancer. Technological advances in RT have occurred in the past 30 years. These advances, such as three-dimensional image guidance, intensity modulation, and robotics, created challenges and opportunities for the next breakthrough, in which artificial intelligence (AI) will possibly play important roles. AI will replace certain repetitive and labor-intensive tasks and improve the accuracy and consistency of others, particularly those with increased complexity because of technological advances. The improvement in efficiency and consistency is important to manage the increasing cancer patient burden to the society. Furthermore, AI may provide new functionalities that facilitate satisfactory RT. The functionalities include superior images for real-time intervention and adaptive and personalized RT. AI may effectively synthesize and analyze big data for such purposes. This review describes the RT workflow and identifies areas, including imaging, treatment planning, quality assurance, and outcome prediction, that benefit from AI. This review primarily focuses on deep-learning techniques, although conventional machine-learning techniques are also mentioned.

Radiomics for the Diagnosis and Differentiation of Pancreatic Cystic Lesions

Radiomics, also known as quantitative imaging or texture analysis, involves extracting a large number of features traditionally unmeasured in conventional radiological cross-sectional images and converting them into mathematical models. This review describes this approach and its use in the evaluation of pancreatic cystic lesions (PCLs). This discipline has the potential of more accurately assessing, classifying, risk stratifying, and guiding the management of PCLs. Existing studies have provided important insight into the role of radiomics in managing PCLs. Although these studies are limited by the use of retrospective design, single center data, and small sample sizes, radiomic features in combination with clinical data appear to be superior to the current standard of care in differentiating cyst type and in identifying mucinous PCLs with high-grade dysplasia. Combining radiomic features with other novel endoscopic diagnostics, including cyst fluid molecular analysis and confocal endomicroscopy, can potentially optimize the predictive accuracy of these models. There is a need for multicenter prospective studies to elucidate the role of radiomics in the management of PCLs.

Neuro-fuzzy patch-wise R-CNN for multiple sclerosis segmentation

The segmentation of the lesion plays a core role in diagnosis and monitoring of multiple sclerosis (MS). Magnetic resonance imaging (MRI) is the most frequent image modality used to evaluate such lesions. Because of the massive amount of data, manual segmentation cannot be achieved within a sensible time that restricts the usage of accurate quantitative measurement in clinical practice. Therefore, the need for effective automated segmentation techniques is critical. However, a large spatial variability between the structure of brain lesions makes it more challenging. Recently, convolutional neural network (CNN), in particular, the region-based CNN (R-CNN), have attained tremendous progress within the field of object recognition because of its ability to learn and represent features. CNN has proven a last-breaking performance in various fields, such as object recognition, and has also gained more attention in brain imaging, especially in tissue and brain segmentation. In this paper, an automated technique for MS lesion segmentation is proposed, which is built on a 3D patch-wise R-CNN. The proposed system includes two stages: first, segmenting MS lesions in T2-w and FLAIR sequences using R-CNN, then an adaptive neuro-fuzzy inference system (ANFIS) is applied to fuse the results of the two modalities. To evaluate the performance of the proposed method, the public MICCAI2008 MS challenge dataset is employed to segment MS lesions. The experimental results show competitive results of the proposed method compared with the state-of-the-art MS lesion segmentation methods with an average total score of 83.25 and an average sensitivity of 61.8% on the MICCAI2008 testing set. Graphical Abstract The proposed system overview. First, the input of two modalities FLAIR and T2 are pre-processed to remove the skull and correct the bias field. Then 3D patches for lesion and non-lesion tissues are extracted and fed to R-CNN. Each R-CNN produces a probability map of the segmentation result that provides to ANFIS to fuse the results and obtain the final MS lesion segmentation. The MS lesions are shown on a pre-processed FLAIR image.

Unpaired Multi-Modal Segmentation via Knowledge Distillation

Multi-modal learning is typically performed with network architectures containing modality-specific layers and shared layers, utilizing co-registered images of different modalities. We propose a novel learning scheme for unpaired cross-modality image segmentation, with a highly compact architecture achieving superior segmentation accuracy. In our method, we heavily reuse network parameters, by sharing all convolutional kernels across CT and MRI, and only employ modality-specific internal normalization layers which compute respective statistics. To effectively train such a highly compact model, we introduce a novel loss term inspired by knowledge distillation, by explicitly constraining the KL-divergence of our derived prediction distributions between modalities. We have extensively validated our approach on two multi-class segmentation problems: i) cardiac structure segmentation, and ii) abdominal organ segmentation. Different network settings, i.e., 2D dilated network and 3D U-net, are utilized to investigate our method's general efficacy. Experimental results on both tasks demonstrate that our novel multi-modal learning scheme consistently outperforms single-modal training and previous multi-modal approaches.

AssemblyNet: A large ensemble of CNNs for 3D whole brain MRI segmentation

Whole brain segmentation of fine-grained structures using deep learning (DL) is a very challenging task since the number of anatomical labels is very high compared to the number of available training images. To address this problem, previous DL methods proposed to use a single convolution neural network (CNN) or few independent CNNs. In this paper, we present a novel ensemble method based on a large number of CNNs processing different overlapping brain areas. Inspired by parliamentary decision-making systems, we propose a framework called AssemblyNet, made of two "assemblies" of U-Nets. Such a parliamentary system is capable of dealing with complex decisions, unseen problem and reaching a relevant consensus. AssemblyNet introduces sharing of knowledge among neighboring U-Nets, an "amendment" procedure made by the second assembly at higher-resolution to refine the decision taken by the first one, and a final decision obtained by majority voting. During our validation, AssemblyNet showed competitive performance compared to state-of-the-art methods such as U-Net, Joint label fusion and SLANT. Moreover, we investigated the scan-rescan consistency and the robustness to disease effects of our method. These experiences demonstrated the reliability of AssemblyNet. Finally, we showed the interest of using semi-supervised learning to improve the performance of our method.

Evaluation of Deep Neural Networks for Semantic Segmentation of Prostate in T2W MRI

In this paper, we present an evaluation of four encoder–decoder CNNs in the segmentation of the prostate gland in T2W magnetic resonance imaging (MRI) image. The four selected CNNs are FCN, SegNet, U-Net, and DeepLabV3+, which was originally proposed for the segmentation of road scene, biomedical, and natural images. Segmentation of prostate in T2W MRI images is an important step in the automatic diagnosis of prostate cancer to enable better lesion detection and staging of prostate cancer. Therefore, many research efforts have been conducted to improve the segmentation of the prostate gland in MRI images. The main challenges of prostate gland segmentation are blurry prostate boundary and variability in prostate anatomical structure. In this work, we investigated the performance of encoder–decoder CNNs for segmentation of prostate gland in T2W MRI. Image pre-processing techniques including image resizing, center-cropping and intensity normalization are applied to address the issues of inter-patient and inter-scanner variability as well as the issue of dominating background pixels over prostate pixels. In addition, to enrich the network with more data, to increase data variation, and to improve its accuracy, patch extraction and data augmentation are applied prior to training the networks. Furthermore, class weight balancing is used to avoid having biased networks since the number of background pixels is much higher than the prostate pixels. The class imbalance problem is solved by utilizing weighted cross-entropy loss function during the training of the CNN model. The performance of the CNNs is evaluated in terms of the Dice similarity coefficient (DSC) and our experimental results show that patch-wise DeepLabV3+ gives the best performance with DSC equal to 92.8%. This value is the highest DSC score compared to the FCN, SegNet, and U-Net that also competed the recently published state-of-the-art method of prostate segmentation.

Machine learning techniques for biomedical image segmentation: An overview of technical aspects and introduction to state-of-art applications

In recent years, significant progress has been made in developing more accurate and efficient machine learning algorithms for segmentation of medical and natural images. In this review article, we highlight the imperative role of machine learning algorithms in enabling efficient and accurate segmentation in the field of medical imaging. We specifically focus on several key studies pertaining to the application of machine learning methods to biomedical image segmentation. We review classical machine learning algorithms such as Markov random fields, k-means clustering, random forest, etc. Although such classical learning models are often less accurate compared to the deep-learning techniques, they are often more sample efficient and have a less complex structure. We also review different deep-learning architectures, such as the artificial neural networks (ANNs), the convolutional neural networks (CNNs), and the recurrent neural networks (RNNs), and present the segmentation results attained by those learning models that were published in the past 3 yr. We highlight the successes and limitations of each machine learning paradigm. In addition, we discuss several challenges related to the training of different machine learning models, and we present some heuristics to address those challenges.

Generating segmentation masks of herbarium specimens and a data set for training segmentation models using deep learning

PREMISE

Digitized images of herbarium specimens are highly diverse with many potential sources of visual noise and bias. The systematic removal of noise and minimization of bias must be achieved in order to generate biological insights based on the plants rather than the digitization and mounting practices involved. Here, we develop a workflow and data set of high-resolution image masks to segment plant tissues in herbarium specimen images and remove background pixels using deep learning.

METHODS AND RESULTS

We generated 400 curated, high-resolution masks of ferns using a combination of automatic and manual tools for image manipulation. We used those images to train a U-Net-style deep learning model for image segmentation, achieving a final Sørensen-Dice coefficient of 0.96. The resulting model can automatically, efficiently, and accurately segment massive data sets of digitized herbarium specimens, particularly for ferns.

CONCLUSIONS

The application of deep learning in herbarium sciences requires transparent and systematic protocols for generating training data so that these labor-intensive resources can be generalized to other deep learning applications. Segmentation ground-truth masks are hard-won data, and we share these data and the model openly in the hopes of furthering model training and transfer learning opportunities for broader herbarium applications.

Deep learning in generating radiology reports: A survey

Substantial progress has been made towards implementing automated radiology reporting models based on deep learning (DL). This is due to the introduction of large medical text/image datasets. Generating radiology coherent paragraphs that do more than traditional medical image annotation, or single sentence-based description, has been the subject of recent academic attention. This presents a more practical and challenging application and moves towards bridging visual medical features and radiologist text. So far, the most common approach has been to utilize publicly available datasets and develop DL models that integrate convolutional neural networks (CNN) for image analysis alongside recurrent neural networks (RNN) for natural language processing (NLP) and natural language generation (NLG). This is an area of research that we anticipate will grow in the near future. We focus our investigation on the following critical challenges: understanding radiology text/image structures and datasets, applying DL algorithms (mainly CNN and RNN), generating radiology text, and improving existing DL based models and evaluation metrics. Lastly, we include a critical discussion and future research recommendations. This survey will be useful for researchers interested in DL, particularly those interested in applying DL to radiology reporting.

A 3D Fully Convolutional Neural Network With Top-Down Attention-Guided Refinement for Accurate and Robust Automatic Segmentation of Amygdala and Its Subnuclei

Recent advances in deep learning have improved the segmentation accuracy of subcortical brain structures, which would be useful in neuroimaging studies of many neurological disorders. However, most existing deep learning based approaches in neuroimaging do not investigate the specific difficulties that exist in segmenting extremely small but important brain regions such as the subnuclei of the amygdala. To tackle this challenging task, we developed a dual-branch dilated residual 3D fully convolutional network with parallel convolutions to extract more global context and alleviate the class imbalance issue by maintaining a small receptive field that is just the size of the regions of interest (ROIs). We also conduct multi-scale feature fusion in both parallel and series to compensate the potential information loss during convolutions, which has been shown to be important for small objects. The serial feature fusion enabled by residual connections is further enhanced by a proposed top-down attention-guided refinement unit, where the high-resolution low-level spatial details are selectively integrated to complement the high-level but coarse semantic information, enriching the final feature representations. As a result, the segmentations resulting from our method are more accurate both volumetrically and morphologically, compared with other deep learning based approaches. To the best of our knowledge, this work is the first deep learning-based approach that targets the subregions of the amygdala. We also demonstrated the feasibility of using a cycle-consistent generative adversarial network (CycleGAN) to harmonize multi-site MRI data, and show that our method generalizes well to challenging traumatic brain injury (TBI) datasets collected from multiple centers. This appears to be a promising strategy for image segmentation for multiple site studies and increased morphological variability from significant brain pathology.

Fast and Precise Hippocampus Segmentation Through Deep Convolutional Neural Network Ensembles and Transfer Learning

Automatic segmentation of the hippocampus from 3D magnetic resonance imaging mostly relied on multi-atlas registration methods. In this work, we exploit recent advances in deep learning to design and implement a fully automatic segmentation method, offering both superior accuracy and fast result. The proposed method is based on deep Convolutional Neural Networks (CNNs) and incorporates distinct segmentation and error correction steps. Segmentation masks are produced by an ensemble of three independent models, operating with orthogonal slices of the input volume, while erroneous labels are subsequently corrected by a combination of Replace and Refine networks. We explore different training approaches and demonstrate how, in CNN-based segmentation, multiple datasets can be effectively combined through transfer learning techniques, allowing for improved segmentation quality. The proposed method was evaluated using two different public datasets and compared favorably to existing methodologies. In the EADC-ADNI HarP dataset, the correspondence between the method's output and the available ground truth manual tracings yielded a mean Dice value of 0.9015, while the required segmentation time for an entire MRI volume was 14.8 seconds. In the MICCAI dataset, the mean Dice value increased to 0.8835 through transfer learning from the larger EADC-ADNI HarP dataset.

Classification of COVID-19 patients from chest CT images using multi-objective differential evolution-based convolutional neural networks

Early classification of 2019 novel coronavirus disease (COVID-19) is essential for disease cure and control. Compared with reverse-transcription polymerase chain reaction (RT-PCR), chest computed tomography (CT) imaging may be a significantly more trustworthy, useful, and rapid technique to classify and evaluate COVID-19, specifically in the epidemic region. Almost all hospitals have CT imaging machines; therefore, the chest CT images can be utilized for early classification of COVID-19 patients. However, the chest CT-based COVID-19 classification involves a radiology expert and considerable time, which is valuable when COVID-19 infection is growing at rapid rate. Therefore, an automated analysis of chest CT images is desirable to save the medical professionals' precious time. In this paper, a convolutional neural networks (CNN) is used to classify the COVID-19-infected patients as infected (+ve) or not (-ve). Additionally, the initial parameters of CNN are tuned using multi-objective differential evolution (MODE). Extensive experiments are performed by considering the proposed and the competitive machine learning techniques on the chest CT images. Extensive analysis shows that the proposed model can classify the chest CT images at a good accuracy rate.