Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection

PARADISE: Personalized and regional adaptation for HIE disease identification and segmentation

Hypoxic ischemic encephalopathy (HIE) is a brain dysfunction occurring in approximately 1-5/1000 term-born neonates. Accurate segmentation of HIE lesions in brain MRI is crucial for prognosis and diagnosis but presents a unique challenge due to the diffuse and small nature of these abnormalities, which resulted in a substantial gap between the performance of machine learning-based segmentation methods and clinical expert annotations for HIE. To address this challenge, we introduce ParadiseNet, an algorithm specifically designed for HIE lesion segmentation. ParadiseNet incorporates global-local learning, progressive uncertainty learning, and self-evolution learning modules, all inspired by clinical interpretation of neonatal brain MRIs. These modules target issues such as unbalanced data distribution, boundary uncertainty, and imprecise lesion detection, respectively. Extensive experiments demonstrate that ParadiseNet significantly enhances small lesion detection (<1%) accuracy in HIE, achieving an over 4% improvement in Dice, 6% improvement in NSD compared to U-Net and other general medical image segmentation algorithms.

Deep Learning techniques to detect and analysis of multiple sclerosis through MRI: A systematic literature review

Deep learning (DL) techniques represent a rapidly advancing field within artificial intelligence, gaining significant prominence in the detection and analysis of various medical conditions through the analysis of medical data. This study presents a systematic literature review (SLR) focused on deep learning methods for the detection and analysis of multiple sclerosis (MS) using magnetic resonance imaging (MRI). The initial search identified 401 articles, which were rigorously screened, a selection of 82 highly relevant studies. These selected studies primarily concentrate on key areas such as multiple sclerosis, deep learning, convolutional neural networks (CNN), lesion segmentation, and classification, reflecting their alignment with the current state of the art. This review comprehensively examines diverse deep-learning approaches for MS detection and analysis, offering a valuable resource for researchers. Additionally, it presents key insights by summarizing these DL techniques for MS detection and analysis using MRI in a structured tabular format.

Machine learning for refining interpretation of magnetic resonance imaging scans in the management of multiple sclerosis: a narrative review

Multiple sclerosis (MS) is an autoimmune disease of the brain and spinal cord with both inflammatory and neurodegenerative features. Although advances in imaging techniques, particularly magnetic resonance imaging (MRI), have improved the process of diagnosis, its cause is unknown, a cure remains elusive and the evidence base to guide treatment is lacking. Computational techniques like machine learning (ML) have started to be used to understand MS. Published MS MRI-based computational studies can be divided into five categories: automated diagnosis; differentiation between lesion types and/or MS stages; differential diagnosis; monitoring and predicting disease progression; and synthetic MRI dataset generation. Collectively, these approaches show promise in assisting with MS diagnosis, monitoring of disease activity and prediction of future progression, all potentially contributing to disease management. Analysis quality using ML is highly dependent on the dataset size and variability used for training. Wider public access would mean larger datasets for experimentation, resulting in higher-quality analysis, permitting for more conclusive research. This narrative review provides an outline of the fundamentals of MS pathology and pathogenesis, diagnostic techniques and data types in computational analysis, as well as collating literature pertaining to the application of computational techniques to MRI towards developing a better understanding of MS.

Accumulated seizure burden predicts neurodevelopmental outcome at 36 months of age in patients with tuberous sclerosis complex

OBJECTIVE

Epilepsy and intellectual disability are common in tuberous sclerosis complex (TSC). Although early life seizures and intellectual disability are known to be correlated in TSC, the differential effects of age at seizure onset and accumulated seizure burden on development remain unclear.

METHODS

Daily seizure diaries, serial neurodevelopmental testing, and brain magnetic resonance imaging were analyzed for 129 TSC patients followed from 0 to 36 months. We used machine learning to identify subgroups of patients based on neurodevelopmental test scores at 36 months of age and assessed the stability of those subgroups at 12 months. We tested the ability of candidate biomarkers to predict 36-month neurodevelopmental subgroup using univariable and multivariable logistic regression. Candidate biomarkers included age at seizure onset, accumulated seizure burden, tuber volume, sex, and earlier neurodevelopmental test scores.

RESULTS

Patients clustered into two neurodevelopmental subgroups at 36 months of age, higher and lower scoring. Subgroup was mostly (75%) the same at 12 months. Significant univariable effects on subgroup were seen only for accumulated seizure burden (largest effect), earlier test scores, and tuber volume. Neither age at seizure onset nor sex significantly distinguished 36-month subgroups, although for girls but not boys there was a significant effect of age at seizure onset. In the multivariable model, accumulated seizure burden and earlier test scores together predicted 36-month neurodevelopmental group with 82% accuracy and an area under the curve of .86.

SIGNIFICANCE

These results untangle the contributions of age at seizure onset and accumulated seizure burden to neurodevelopmental outcomes in young children with TSC. Accumulated seizure burden, rather than the age at seizure onset, most accurately predicts neurodevelopmental outcome at 36 months of age. These results emphasize the need to manage seizures aggressively during the first 3 years of life for patients with TSC, not only to promote seizure control but to optimize cognitive function.

Medical Image Segmentation Review: The Success of U-Net

Automatic medical image segmentation is a crucial topic in the medical domain and successively a critical counterpart in the computer-aided diagnosis paradigm. U-Net is the most widespread image segmentation architecture due to its flexibility, optimized modular design, and success in all medical image modalities. Over the years, the U-Net model has received tremendous attention from academic and industrial researchers who have extended it to address the scale and complexity created by medical tasks. These extensions are commonly related to enhancing the U-Net's backbone, bottleneck, or skip connections, or including representation learning, or combining it with a Transformer architecture, or even addressing probabilistic prediction of the segmentation map. Having a compendium of different previously proposed U-Net variants makes it easier for machine learning researchers to identify relevant research questions and understand the challenges of the biological tasks that challenge the model. In this work, we discuss the practical aspects of the U-Net model and organize each variant model into a taxonomy. Moreover, to measure the performance of these strategies in a clinical application, we propose fair evaluations of some unique and famous designs on well-known datasets. Furthermore, we provide a comprehensive implementation library with trained models. In addition, for ease of future studies, we created an online list of U-Net papers with their possible official implementation.

Hyperparameter selection for dataset-constrained semantic segmentation: Practical machine learning optimization

PURPOSE/AIM

This paper provides a pedagogical example for systematic machine learning optimization in small dataset image segmentation, emphasizing hyperparameter selections. A simple process is presented for medical physicists to examine hyperparameter optimization. This is also applied to a case-study, demonstrating the benefit of the method.

MATERIALS AND METHODS

An unrestricted public Computed Tomography (CT) dataset, with binary organ segmentation, was used to develop a multiclass segmentation model. To start the optimization process, a preliminary manual search of hyperparameters was conducted and from there a grid search identified the most influential result metrics. A total of 658 different models were trained in 2100 h, using 13 160 effective patients. The quantity of results was analyzed using random forest regression, identifying relative hyperparameter impact.

RESULTS

Metric implied segmentation quality (accuracy 96.8%, precision 95.1%) and visual inspection were found to be mismatched. In this work batch normalization was most important, but performance varied with hyperparameters and metrics selected. Targeted grid-search optimization and random forest analysis of relative hyperparameter importance, was an easily implementable sensitivity analysis approach.

CONCLUSION

The proposed optimization method gives a systematic and quantitative approach to something intuitively understood, that hyperparameters change model performance. Even just grid search optimization with random forest analysis presented here can be informative within hardware and data quality/availability limitations, adding confidence to model validity and minimize decision-making risks. By providing a guided methodology, this work helps medical physicists to improve their model optimization, irrespective of specific challenges posed by datasets and model design.

Deep geometric learning for intracranial aneurysm detection: towards expert rater performance

BACKGROUND

Early detection of intracranial aneurysms (IAs) is crucial for patient outcomes. Typically identified on angiographic scans such as CT angiography (CTA) or MR angiography (MRA), the sensitivity of experts in studies on small IAs (diameter <3 mm) was moderate (64-74.1% for CTAs and 70-92.8% for MRAs), and these figures could be lower in a routine clinical setting. Recent research shows that the expert level of sensitivity might be achieved using deep learning approaches.

METHODS

A large multisite dataset including 1054 MRA and 2174 CTA scans with expert IA annotations was collected. A novel modality-agnostic two-step IA detection approach was proposed. The first step used nnU-Net for segmenting vascular structures, with model training performed separately for each modality. In the second step, segmentations were converted to vascular surface that was parcellated by sampling point clouds and, using a PointNet++ model, each point was labeled as an aneurysm or vessel class.

RESULTS

Quantitative validation of the test data from different sites than the training data showed that the proposed approach achieved pooled sensitivity of 85% and 90% on 157 MRA scans and 1338 CTA scans, respectively, while the sensitivity for small IAs was 72% and 83%, respectively. The corresponding number of false findings per image was low at 1.54 and 1.57, and 0.4 and 0.83 on healthy subject data.

CONCLUSIONS

The proposed approach achieved a state-of-the-art balance between the sensitivity and the number of false findings, matched the expert-level sensitivity to small (and other) IAs on external data, and therefore seems fit for computer-assisted detection of IAs in a clinical setting.

Crispr-SGRU: Prediction of CRISPR/Cas9 Off-Target Activities with Mismatches and Indels Using Stacked BiGRU

CRISPR/Cas9 is a popular genome editing technology, yet its clinical application is hindered by off-target effects. Many deep learning-based methods are available for off-target prediction. However, few can predict off-target activities with insertions or deletions (indels) between single guide RNA and DNA sequence pairs. Additionally, the analysis of off-target data is challenged due to a data imbalance issue. Moreover, the prediction accuracy and interpretability remain to be improved. Here, we introduce a deep learning-based framework, named Crispr-SGRU, to predict off-target activities with mismatches and indels. This model is based on Inception and stacked BiGRU. It adopts a dice loss function to solve the inherent imbalance issue. Experimental results show our model outperforms existing methods for off-target prediction in terms of accuracy and robustness. Finally, we study the interpretability of this model through Deep SHAP and teacher-student-based knowledge distillation, and find it can provide meaningful explanations for sequence patterns regarding off-target activity.

GREnet: Gradually REcurrent Network With Curriculum Learning for 2-D Medical Image Segmentation

Medical image segmentation is a vital stage in medical image analysis. Numerous deep-learning methods are booming to improve the performance of 2-D medical image segmentation, owing to the fast growth of the convolutional neural network. Generally, the manually defined ground truth is utilized directly to supervise models in the training phase. However, direct supervision of the ground truth often results in ambiguity and distractors as complex challenges appear simultaneously. To alleviate this issue, we propose a gradually recurrent network with curriculum learning, which is supervised by gradual information of the ground truth. The whole model is composed of two independent networks. One is the segmentation network denoted as GREnet, which formulates 2-D medical image segmentation as a temporal task supervised by pixel-level gradual curricula in the training phase. The other is a curriculum-mining network. To a certain degree, the curriculum-mining network provides curricula with an increasing difficulty in the ground truth of the training set by progressively uncovering hard-to-segmentation pixels via a data-driven manner. Given that segmentation is a pixel-level dense-prediction challenge, to the best of our knowledge, this is the first work to function 2-D medical image segmentation as a temporal task with pixel-level curriculum learning. In GREnet, the naive UNet is adopted as the backbone, while ConvLSTM is used to establish the temporal link between gradual curricula. In the curriculum-mining network, UNet++ supplemented by transformer is designed to deliver curricula through the outputs of the modified UNet++ at different layers. Experimental results have demonstrated the effectiveness of GREnet on seven datasets, i.e., three lesion segmentation datasets in dermoscopic images, an optic disc and cup segmentation dataset and a blood vessel segmentation dataset in retinal images, a breast lesion segmentation dataset in ultrasound images, and a lung segmentation dataset in computed tomography (CT).

CACSNet for automatic robust classification and segmentation of carotid artery calcification on panoramic radiographs using a cascaded deep learning network

Stroke is one of the major causes of death worldwide, and is closely associated with atherosclerosis of the carotid artery. Panoramic radiographs (PRs) are routinely used in dental practice, and can be used to visualize carotid artery calcification (CAC). The purpose of this study was to automatically and robustly classify and segment CACs with large variations in size, shape, and location, and those overlapping with anatomical structures based on deep learning analysis of PRs. We developed a cascaded deep learning network (CACSNet) consisting of classification and segmentation networks for CACs on PRs. This network was trained on ground truth data accurately determined with reference to CT images using the Tversky loss function with optimized weights by balancing between precision and recall. CACSNet with EfficientNet-B4 achieved an AUC of 0.996, accuracy of 0.985, sensitivity of 0.980, and specificity of 0.988 in classification for normal or abnormal PRs. Segmentation performances for CAC lesions were 0.595 for the Jaccard index, 0.722 for the Dice similarity coefficient, 0.749 for precision, and 0.756 for recall. Our network demonstrated superior classification performance to previous methods based on PRs, and had comparable segmentation performance to studies based on other imaging modalities. Therefore, CACSNet can be used for robust classification and segmentation of CAC lesions that are morphologically variable and overlap with surrounding structures over the entire posterior inferior region of the mandibular angle on PRs.

Teeth segmentation and carious lesions segmentation in panoramic X-ray images using CariSeg, a networks' ensemble

Background

Dental cavities are common oral diseases that can lead to pain, discomfort, and eventually, tooth loss. Early detection and treatment of cavities can prevent these negative consequences. We propose CariSeg, an intelligent system composed of four neural networks that result in the detection of cavities in dental X-rays with 99.42% accuracy.

Method

The first model of CariSeg, trained using the U-Net architecture, segments the area of interest, the teeth, and crops the radiograph around it. The next component segments the carious lesions and it is an ensemble composed of three architectures: U-Net, Feature Pyramid Network, and DeeplabV3. For tooth identification two merged datasets were used: The Tufts Dental Database consisting of 1000 panoramic radiography images and another dataset of 116 anonymized panoramic X-rays, taken at Noor Medical Imaging Center, Qom. For carious lesion segmentation, a dataset consisting of 150 panoramic X-ray images was acquired from the Department of Oral and Maxillofacial Surgery and Radiology, Iuliu Hatieganu University of Medicine and Pharmacy, Cluj-Napoca.

Results

The experiments demonstrate that our approach results in 99.42% accuracy and a mean 68.2% Dice coefficient.

Conclusions

AI helps in detecting carious lesions by analyzing dental X-rays and identifying cavities that might be missed by human observers, leading to earlier detection and treatment of cavities and resulting in better oral health outcomes.

The macular retinal ganglion cell layer as a biomarker for diagnosis and prognosis in multiple sclerosis: A deep learning approach

PURPOSE

The macular ganglion cell layer (mGCL) is a strong potential biomarker of axonal degeneration in multiple sclerosis (MS). For this reason, this study aims to develop a computer-aided method to facilitate diagnosis and prognosis in MS.

METHODS

This paper combines a cross-sectional study of 72 MS patients and 30 healthy control subjects for diagnosis and a 10-year longitudinal study of the same MS patients for the prediction of disability progression, during which the mGCL was measured using optical coherence tomography (OCT). Deep neural networks were used as an automatic classifier.

RESULTS

For MS diagnosis, greatest accuracy (90.3%) was achieved using 17 features as inputs. The neural network architecture comprised the input layer, two hidden layers and the output layer with softmax activation. For the prediction of disability progression 8 years later, accuracy of 81.9% was achieved with a neural network comprising two hidden layers and 400 epochs.

CONCLUSION

We present evidence that by applying deep learning techniques to clinical and mGCL thickness data it is possible to identify MS and predict the course of the disease. This approach potentially constitutes a non-invasive, low-cost, easy-to-implement and effective method.

Medical long-tailed learning for imbalanced data: Bibliometric analysis

BACKGROUND

In the last decade, long-tail learning has become a popular research focus in deep learning applications in medicine. However, no scientometric reports have provided a systematic overview of this scientific field. We utilized bibliometric techniques to identify and analyze the literature on long-tailed learning in deep learning applications in medicine and investigate research trends, core authors, and core journals. We expanded our understanding of the primary components and principal methodologies of long-tail learning research in the medical field.

METHODS

Web of Science was utilized to collect all articles on long-tailed learning in medicine published until December 2023. The suitability of all retrieved titles and abstracts was evaluated. For bibliometric analysis, all numerical data were extracted. CiteSpace was used to create clustered and visual knowledge graphs based on keywords.

RESULTS

A total of 579 articles met the evaluation criteria. Over the last decade, the annual number of publications and citation frequency both showed significant growth, following a power-law and exponential trend, respectively. Noteworthy contributors to this field include Husanbir Singh Pannu, Fadi Thabtah, and Talha Mahboob Alam, while leading journals such as IEEE ACCESS, COMPUTERS IN BIOLOGY AND MEDICINE, IEEE TRANSACTIONS ON MEDICAL IMAGING, and COMPUTERIZED MEDICAL IMAGING AND GRAPHICS have emerged as pivotal platforms for disseminating research in this area. The core of long-tailed learning research within the medical domain is encapsulated in six principal themes: deep learning for imbalanced data, model optimization, neural networks in image analysis, data imbalance in health records, CNN in diagnostics and risk assessment, and genetic information in disease mechanisms.

CONCLUSION

This study summarizes recent advancements in applying long-tail learning to deep learning in medicine through bibliometric analysis and visual knowledge graphs. It explains new trends, sources, core authors, journals, and research hotspots. Although this field has shown great promise in medical deep learning research, our findings will provide pertinent and valuable insights for future research and clinical practice.

Evaluation of the quality and the productivity of neuroradiological reading of multiple sclerosis follow-up MRI scans using an intelligent automation software

PURPOSE

The assessment of multiple sclerosis (MS) lesions on follow-up magnetic resonance imaging (MRI) is tedious, time-consuming, and error-prone. Automation of low-level tasks could enhance the radiologist in this work. We evaluate the intelligent automation software Jazz in a blinded three centers study, for the assessment of new, slowly expanding, and contrast-enhancing MS lesions.

METHODS

In three separate centers, 117 MS follow-up MRIs were blindly analyzed on fluid attenuated inversion recovery (FLAIR), pre- and post-gadolinium T1-weighted images using Jazz by 2 neuroradiologists in each center. The reading time was recorded. The ground truth was defined in a second reading by side-by-side comparison of both reports from Jazz and the standard clinical report. The number of described new, slowly expanding, and contrast-enhancing lesions described with Jazz was compared to the lesions described in the standard clinical report.

RESULTS

A total of 96 new lesions from 41 patients and 162 slowly expanding lesions (SELs) from 61 patients were described in the ground truth reading. A significantly larger number of new lesions were described using Jazz compared to the standard clinical report (63 versus 24). No SELs were reported in the standard clinical report, while 95 SELs were reported on average using Jazz. A total of 4 new contrast-enhancing lesions were found in all reports. The reading with Jazz was very time efficient, taking on average 2min33s ± 1min0s per case. Overall inter-reader agreement for new lesions between the readers using Jazz was moderate for new lesions (Cohen kappa = 0.5) and slight for SELs (0.08).

CONCLUSION

The quality and the productivity of neuroradiological reading of MS follow-up MRI scans can be significantly improved using the dedicated software Jazz.

Investigation of an efficient multi-modal convolutional neural network for multiple sclerosis lesion detection

In this study, an automated 2D machine learning approach for fast and precise segmentation of MS lesions from multi-modal magnetic resonance images (mmMRI) is presented. The method is based on an U-Net like convolutional neural network (CNN) for automated 2D slice-based-segmentation of brain MRI volumes. The individual modalities are encoded in separate downsampling branches without weight sharing, to leverage the specific features. Skip connections input feature maps to multi-scale feature fusion (MSFF) blocks at every decoder stage of the network. Those are followed by multi-scale feature upsampling (MSFU) blocks which use the information about lesion shape and location. The CNN is evaluated on two publicly available datasets: The ISBI 2015 longitudinal MS lesion segmentation challenge dataset containing 19 subjects and the MICCAI 2016 MSSEG challenge dataset containing 15 subjects from various scanners. The proposed multi-input 2D architecture is among the top performing approaches in the ISBI challenge, to which open-access papers are available, is able to outperform state-of-the-art 3D approaches without additional post-processing, can be adapted to other scanners quickly, is robust against scanner variability and can be deployed for inference even on a standard laptop without a dedicated GPU.

Loss-Modified Transformer-Based U-Net for Accurate Segmentation of Fluids in Optical Coherence Tomography Images of Retinal Diseases

Background

Optical coherence tomography (OCT) imaging significantly contributes to ophthalmology in the diagnosis of retinal disorders such as age-related macular degeneration and diabetic macular edema. Both diseases involve the abnormal accumulation of fluids, location, and volume, which is vitally informative in detecting the severity of the diseases. Automated and accurate fluid segmentation in OCT images could potentially improve the current clinical diagnosis. This becomes more important by considering the limitations of manual fluid segmentation as a time-consuming and subjective to error method.

Methods

Deep learning techniques have been applied to various image processing tasks, and their performance has already been explored in the segmentation of fluids in OCTs. This article suggests a novel automated deep learning method utilizing the U-Net structure as the basis. The modifications consist of the application of transformers in the encoder path of the U-Net with the purpose of more concentrated feature extraction. Furthermore, a custom loss function is empirically tailored to efficiently incorporate proper loss functions to deal with the imbalance and noisy images. A weighted combination of Dice loss, focal Tversky loss, and weighted binary cross-entropy is employed.

Results

Different metrics are calculated. The results show high accuracy (Dice coefficient of 95.52) and robustness of the proposed method in comparison to different methods after adding extra noise to the images (Dice coefficient of 92.79).

Conclusions

The segmentation of fluid regions in retinal OCT images is critical because it assists clinicians in diagnosing macular edema and executing therapeutic operations more quickly. This study suggests a deep learning framework and novel loss function for automated fluid segmentation of retinal OCT images with excellent accuracy and rapid convergence result.

How far MS lesion detection and segmentation are integrated into the clinical workflow? A systematic review

INTRODUCTION

Over the past few years, the deep learning community has developed and validated a plethora of tools for lesion detection and segmentation in Multiple Sclerosis (MS). However, there is an important gap between validating models technically and clinically. To this end, a six-step framework necessary for the development, validation, and integration of quantitative tools in the clinic was recently proposed under the name of the Quantitative Neuroradiology Initiative (QNI).

AIMS

Investigate to what extent automatic tools in MS fulfill the QNI framework necessary to integrate automated detection and segmentation into the clinical neuroradiology workflow.

METHODS

Adopting the systematic Cochrane literature review methodology, we screened and summarised published scientific articles that perform automatic MS lesions detection and segmentation. We categorised the retrieved studies based on their degree of fulfillment of QNI's six-steps, which include a tool's technical assessment, clinical validation, and integration.

RESULTS

We found 156 studies; 146/156 (94%) fullfilled the first QNI step, 155/156 (99%) the second, 8/156 (5%) the third, 3/156 (2%) the fourth, 5/156 (3%) the fifth and only one the sixth.

CONCLUSIONS

To date, little has been done to evaluate the clinical performance and the integration in the clinical workflow of available methods for MS lesion detection/segmentation. In addition, the socio-economic effects and the impact on patients' management of such tools remain almost unexplored.

Prediction of Primary Tumor Sites in Spinal Metastases Using a ResNet-50 Convolutional Neural Network Based on MRI

We aim to investigate the feasibility and evaluate the performance of a ResNet-50 convolutional neural network (CNN) based on magnetic resonance imaging (MRI) in predicting primary tumor sites in spinal metastases. Conventional sequences (T1-weighted, T2-weighted, and fat-suppressed T2-weighted sequences) MRIs of spinal metastases patients confirmed by pathology from August 2006 to August 2019 were retrospectively analyzed. Patients were partitioned into non-overlapping sets of 90% for training and 10% for testing. A deep learning model using ResNet-50 CNN was trained to classify primary tumor sites. Top-1 accuracy, precision, sensitivity, area under the curve for the receiver-operating characteristic (AUC-ROC), and F1 score were considered as the evaluation metrics. A total of 295 spinal metastases patients (mean age ± standard deviation, 59.9 years ± 10.9; 154 men) were evaluated. Included metastases originated from lung cancer (n = 142), kidney cancer (n = 50), mammary cancer (n = 41), thyroid cancer (n = 34), and prostate cancer (n = 28). For 5-class classification, AUC-ROC and top-1 accuracy were 0.77 and 52.97%, respectively. Additionally, AUC-ROC for different sequence subsets ranged between 0.70 (for T2-weighted) and 0.74 (for fat-suppressed T2-weighted). Our developed ResNet-50 CNN model for predicting primary tumor sites in spinal metastases at MRI has the potential to help prioritize the examinations and treatments in case of unknown primary for radiologists and oncologists.

A Compound Loss Function With Shape Aware Weight Map for Microscopy Cell Segmentation

Microscopy cell segmentation is a crucial step in biological image analysis and a challenging task. In recent years, deep learning has been widely used to tackle this task, with promising results. A critical aspect of training complex neural networks for this purpose is the selection of the loss function, as it affects the learning process. In the field of cell segmentation, most of the recent research in improving the loss function focuses on addressing the problem of inter-class imbalance. Despite promising achievements, more work is needed, as the challenge of cell segmentation is not only the inter-class imbalance but also the intra-class imbalance (the cost imbalance between the false positives and false negatives of the inference model), the segmentation of cell minutiae, and the missing annotations. To deal with these challenges, in this paper, we propose a new compound loss function employing a shape aware weight map. The proposed loss function is inspired by Youden's J index to handle the problem of inter-class imbalance and uses a focal cross-entropy term to penalize the intra-class imbalance and weight easy/hard samples. The proposed shape aware weight map can handle the problem of missing annotations and facilitate valid segmentation of cell minutiae. Results of evaluations on all ten 2D+time datasets from the public cell tracking challenge demonstrate 1) the superiority of the proposed loss function with the shape aware weight map, and 2) that the performance of recent deep learning-based cell segmentation methods can be improved by using the proposed compound loss function.

A GAN-based method for 3D lung tumor reconstruction boosted by a knowledge transfer approach

Three-dimensional (3D) image reconstruction of tumors has been one of the most effective techniques for accurately visualizing tumor structures and treatment with high resolution, which requires a set of two-dimensional medical images such as CT slices. In this paper we propose a novel method based on generative adversarial networks (GANs) for 3D lung tumor reconstruction by CT images. The proposed method consists of three stages: lung segmentation, tumor segmentation and 3D lung tumor reconstruction. Lung and tumor segmentation are performed using snake optimization and Gustafson-Kessel (GK) clustering. In the 3D reconstruction part first, features are extracted using the pre-trained VGG model from the tumors that detected in 2D CT slices. Then, a sequence of extracted features is fed into an LSTM to output compressed features. Finally, the compressed feature is used as input for GAN, where the generator is responsible for high-level reconstructing the 3D image of the lung tumor. The main novelty of this paper is the use of GAN to reconstruct a 3D lung tumor model for the first time, to the best of our knowledge. Also, we used knowledge transfer to extract features from 2D images to speed up the training process. The results obtained from the proposed model on the LUNA dataset showed better results than state of the art. According to HD and ED metrics, the proposed method has the lowest values ​​of 3.02 and 1.06, respectively, as compared to those of other methods. The experimental results show that the proposed method performs better than previous similar methods and it is useful to help practitioners in the treatment process.

A hierarchical GAN method with ensemble CNN for accurate nodule detection

PURPOSE

Lung cancer can evolve into one of the deadliest diseases whose early detection is one of the major survival factors. However, early detection is a challenging task due to the unclear structure, shape, and the size of the nodule. Hence, radiologists need automated tools to make accurate decisions.

METHODS

This paper develops a new approach based on generative adversarial network (GAN) architecture for nodule detection to propose a two-step GAN model containing lung segmentation and nodule localization. The first generator comprises a U-net network, while the second utilizes a mask R-CNN. The task of lung segmentation involves a two-class classification of the pixels in each image, categorizing lung pixels in one class and the rest in the other. The classifier becomes imbalanced due to numerous non-lung pixels, decreasing the model performance. This problem is resolved by using the focal loss function for training the generator. Moreover, a new loss function is developed as the nodule localization generator to enhance the diagnosis quality. Discriminator nets are implemented in GANs as an ensemble of convolutional neural networks (ECNNs), using multiple CNNs and connecting their outputs to make a final decision.

RESULTS

Several experiments are designed to assess the model on the well-known LUNA dataset. The experiments indicate that the proposed model can reduce the error of the state-of-the-art models on the IoU criterion by about 35 and 16% for lung segmentation and nodule localization, respectively.

CONCLUSION

Unlike recent studies, the proposed method considers two loss functions for generators, further promoting the goal achievements. Moreover, the network of discriminators is regarded as ECNNs, generating rich features for decisions.

A Disease-Prediction Protocol Integrating Triage Priority and BERT-Based Transfer Learning for Intelligent Triage

Large hospitals can be complex, with numerous discipline and subspecialty settings. Patients may have limited medical knowledge, making it difficult for them to determine which department to visit. As a result, visits to the wrong departments and unnecessary appointments are common. To address this issue, modern hospitals require a remote system capable of performing intelligent triage, enabling patients to perform self-service triage. To address the challenges outlined above, this study presents an intelligent triage system based on transfer learning, capable of processing multilabel neurological medical texts. The system predicts a diagnosis and corresponding department based on the patient’s input. It utilizes the triage priority (TP) method to label diagnostic combinations found in medical records, converting a multilabel problem into a single-label one. The system considers disease severity and reduces the “class overlapping” of the dataset. The BERT model classifies the chief complaint text, predicting a primary diagnosis corresponding to the complaint. To address data imbalance, a composite loss function based on cost-sensitive learning is added to the BERT architecture. The study results indicate that the TP method achieves a classification accuracy of 87.47% on medical record text, outperforming other problem transformation methods. By incorporating the composite loss function, the system’s accuracy rate improves to 88.38% surpassing other loss functions. Compared to traditional methods, this system does not introduce significant complexity, yet substantially improves triage accuracy, reduces patient input confusion, and enhances hospital triage capabilities, ultimately improving the patient’s medical experience. The findings could provide a reference for intelligent triage development.

Multi-arm U-Net with dense input and skip connectivity for T2 lesion segmentation in clinical trials of multiple sclerosis

T2 lesion quantification plays a crucial role in monitoring disease progression and evaluating treatment response in multiple sclerosis (MS). We developed a 3D, multi-arm U-Net for T2 lesion segmentation, which was trained on a large, multicenter clinical trial dataset of relapsing MS. We investigated its generalization to other relapsing and primary progressive MS clinical trial datasets, and to an external dataset from the MICCAI 2016 MS lesion segmentation challenge. Additionally, we assessed the model's ability to reproduce the separation of T2 lesion volumes between treatment and control arms; and the association of baseline T2 lesion volumes with clinical disability scores compared with manual lesion annotations. The trained model achieved a mean dice coefficient of ≥ 0.66 and a lesion detection sensitivity of ≥ 0.72 across the internal test datasets. On the external test dataset, the model achieved a mean dice coefficient of 0.62, which is comparable to 0.59 from the best model in the challenge, and a lesion detection sensitivity of 0.68. Lesion detection performance was reduced for smaller lesions (≤ 30 μL, 3-10 voxels). The model successfully maintained the separation of the longitudinal changes in T2 lesion volumes between the treatment and control arms. Such tools could facilitate semi-automated MS lesion quantification; and reduce rater burden in clinical trials.

Deep-learning-based ensemble method for fully automated detection of renal masses on magnetic resonance images

Purpose

Accurate detection of small renal masses (SRM) is a fundamental step for automated classification of benign and malignant or indolent and aggressive renal tumors. Magnetic resonance image (MRI) may outperform computed tomography (CT) for SRM subtype differentiation due to improved tissue characterization, but is less explored compared to CT. The objective of this study is to autonomously detect SRM on contrast-enhanced magnetic resonance images (CE-MRI).

Approach

In this paper, we described a novel, fully automated methodology for accurate detection and localization of SRM on CE-MRI. We first determine the kidney boundaries using a U-Net convolutional neural network. We then search for SRM within the localized kidney regions using a mixture-of-experts ensemble model based on the U-Net architecture. Our dataset contained CE-MRI scans of 118 patients with different solid kidney tumor subtypes including renal cell carcinomas, oncocytomas, and fat-poor renal angiomyolipoma. We evaluated the proposed model on the entire CE-MRI dataset using 5-fold cross validation.

Results

The developed algorithm reported a Dice similarity coefficient of 91.20 ± 5.41 % (mean ± standard deviation) for kidney segmentation from 118 volumes consisting of 25,025 slices. Our proposed ensemble model for SRM detection yielded a recall and precision of 86.2% and 83.3% on the entire CE-MRI dataset, respectively.

Conclusions

We described a deep-learning-based method for fully automated SRM detection using CE-MR images, which has not been studied previously. The results are clinically important as SRM localization is a pre-step for fully automated diagnosis of SRM subtypes.

A dense residual U-net for multiple sclerosis lesions segmentation from multi-sequence 3D MR images

Multiple Sclerosis (MS) is an autoimmune disease that causes brain and spinal cord lesions, which magnetic resonance imaging (MRI) can detect and characterize. Recently, deep learning methods have achieved remarkable results in the automated segmentation of MS lesions from MRI data. Hence, this study proposes a novel dense residual U-Net model that combines attention gate (AG), efficient channel attention (ECA), and Atrous Spatial Pyramid Pooling (ASPP) to enhance the performance of the automatic MS lesion segmentation using 3D MRI sequences. First, convolution layers in each block of the U-Net architecture are replaced by residual blocks and connected densely. Then, AGs are exploited to capture salient features passed through the skip connections. The ECA module is appended at the end of each residual block and each downsampling block of U-Net. Later, the bottleneck of U-Net is replaced with the ASSP module to extract multi-scale contextual information. Furthermore, 3D MR images of Fluid Attenuated Inversion Recovery (FLAIR), T1-weighted (T1-w), and T2-weighted (T2-w) are exploited jointly to perform better MS lesion segmentation. The proposed model is validated on the publicly available ISBI2015 and MSSEG2016 challenge datasets. This model produced an ISBI score of 92.75, a mean Dice score of 66.88%, a mean positive predictive value (PPV) of 86.50%, and a mean lesion-wise true positive rate (LTPR) of 60.64% on the ISBI2015 testing set. Also, it achieved a mean Dice score of 67.27%, a mean PPV of 65.19%, and a mean sensitivity of 74.40% on the MSSEG2016 testing set. The results show that the proposed model performs better than the results of some experts and some of the other state-of-the-art methods realized related to this particular subject. Specifically, the best Dice score and the best LTPR are obtained on the ISBI2015 testing set by using the proposed model to segment MS lesions.

Automatic detection of active and inactive multiple sclerosis plaques using the Bayesian approach in susceptibility-weighted imaging

BACKGROUND

Susceptibility-weighted imaging (SWI) is efficient in detecting multiple sclerosis (MS) plaques and evaluating the level of disease activity.

PURPOSE

To automatically detect active and inactive MS plaques in SWI images using a Bayesian approach.

MATERIAL AND METHODS

A 1.5-T scanner was used to evaluate 147 patients with MS. The area of the plaques along with their active or inactive status were automatically identified using a Bayesian approach. Plaques were given an orange color if they were active and a blue color if they were inactive, based on the preset signal intensity.

RESULTS

Experimental findings show that the proposed method has a high accuracy rate of 91% and a sensitivity rate of 76% for identifying the type and area of plaques. Inactive plaques were properly identified in 87% of cases, and active plaques in 76% of cases. The Kappa analysis revealed an 80% agreement between expert diagnoses based on contrast-enhanced and FLAIR images and Bayesian inferences in SWI.

CONCLUSION

The results of our study demonstrated that the proposed method has good accuracy for identifying the MS plaque area as well as for identifying the types of active or inactive plaques in SWI. Therefore, it might be helpful to use the proposed method as a supplemental tool to accelerate the specialist's diagnosis.

Diffusion-derived parameters in lesions, peri-lesion and normal-appearing white matter in multiple sclerosis using tensor, kurtosis and fixel-based analysis

Neuronal damage is the primary cause of long-term disability of multiple sclerosis (MS) patients. Assessment of axonal integrity from diffusion MRI parameters might enable better disease characterisation. 16 diffusion derived measurements from diffusion tensor imaging (DTI), diffusion kurtosis imaging (DKI), and fixel-based analysis (FBA) in lesions, peri-lesion and normal appearing white matter were investigated. Diffusion MRI scans of 11 MS patients were processed to generate DTI, DKI, and FBA images. Fractional anisotropy (FA) and fibre density (FD) were used to assess axonal integrity across brain regions. Subsequently, 359 lesions were identified, and lesion and peri-lesion segmentation was performed using structural T1w, T2w, T2w-FLAIR, and T1w post-contrast MRI. The segmentations were then used to extract 16 diffusion MRI parameters from lesion, peri-lesion, and contralateral normal appearing white matter (NAWM). The measurements for axonal integrity, DTI-FA, DKI-FA, FBA-FD, produced similar results. All diffusion MRI parameters were affected in lesions as compared to NAWM (p < 0.001), confirming loss of axonal integrity in lesions. In peri-lesions, most parameters, except FBA-FD, were also significantly different from NAWM, although the effect size was smaller than in lesions. The reduction in axonal integrity in peri-lesions, despite unaffected fibre density estimates, suggests an effect of Wallerian degeneration.

Limited utility of structural MRI to identify the epileptogenic zone in young children with tuberous sclerosis

BACKGROUND AND PURPOSE

The success of epilepsy surgery in children with tuberous sclerosis complex (TSC) hinges on identification of the epileptogenic zone (EZ). We studied structural MRI markers of epileptogenic lesions in young children with TSC.

METHODS

We included 26 children with TSC who underwent epilepsy surgery before the age of 3 years at five sites, with 12 months or more follow-up. Two neuroradiologists, blinded to surgical outcome data, reviewed 10 candidate lesions on preoperative MRI for characteristics of the tuber (large affected area, calcification, cyst-like properties) and of focal cortical dysplasia (FCD) features (cortical malformation, gray-white matter junction blurring, transmantle sign). They selected lesions suspect for the EZ based on structural MRI, and reselected after unblinding to seizure onset location on electroencephalography (EEG).

RESULTS

None of the tuber characteristics and FCD features were distinctive for the EZ, indicated by resected lesions in seizure-free children. With structural MRI alone, the EZ was identified out of 10 lesions in 31%, and with addition of EEG data, this increased to 48%. However, rates of identification of resected lesions in non-seizure-free children were similar. Across 251 lesions, interrater agreement was moderate for large size (κ = .60), and fair (κ = .24) for all other features.

CONCLUSIONS

In young children with TSC, the utility of structural MRI features is limited in the identification of the epileptogenic tuber, but improves when combined with EEG data.

White Matter Lesion Segmentation for Multiple Sclerosis Patients implementing deep learning

The aim of this work is to address the problem of White Matter Lesion (WML) segmentation employing Magnetic Resonance Imaging (MRI) images from Multiple Sclerosis (MS) patients through the application of deep learning. A U-net based architecture containing a contrastive path and an expanding path prior to the final pixel-wise classification is implemented. The data are provided by the Ippokratio Radiology Center of Ioannina and include Fluid-Attenuated Inversion Recovery (FLAIR) MRI images from 30 patients in three phases, baseline and two follow ups. The prediction results are quite significant in terms of pixel-wise classification. The implemented deep learning model demonstrates Dice coefficient 0.7292, Precision 75.92% and Recall 70.16% in 2D slices of FLAIR MRI non-skull stripped images.

Jointly Optimized Spatial Histogram UNET Architecture (JOSHUA) for Adipose Tissue Segmentation

Objective. We aim to develop a machine learning algorithm to quantify adipose tissue deposition at surgical sites as a function of biomaterial implantation. Impact Statement. To our knowledge, this study is the first investigation to apply convolutional neural network (CNN) models to identify and segment adipose tissue in histological images from silk fibroin biomaterial implants. Introduction. When designing biomaterials for the treatment of various soft tissue injuries and diseases, one must consider the extent of adipose tissue deposition. In this work, we analyzed adipose tissue accumulation in histological images of sectioned silk fibroin-based biomaterials excised from rodents following subcutaneous implantation for 1, 2, 4, or 8 weeks. Current strategies for quantifying adipose tissue after biomaterial implantation are often tedious and prone to human bias during analysis. Methods. We used CNN models with novel spatial histogram layer(s) that can more accurately identify and segment regions of adipose tissue in hematoxylin and eosin (H&E) and Masson's trichrome stained images, allowing for determination of the optimal biomaterial formulation. We compared the method, Jointly Optimized Spatial Histogram UNET Architecture (JOSHUA), to the baseline UNET model and an extension of the baseline model, attention UNET, as well as to versions of the models with a supplemental attention-inspired mechanism (JOSHUA+ and UNET+). Results. The inclusion of histogram layer(s) in our models shows improved performance through qualitative and quantitative evaluation. Conclusion. Our results demonstrate that the proposed methods, JOSHUA and JOSHUA+, are highly beneficial for adipose tissue identification and localization. The new histological dataset and code used in our experiments are publicly available.

Multiple Sclerosis Lesions Segmentation Using Attention-Based CNNs in FLAIR Images

Objective: Multiple Sclerosis (MS) is an autoimmune and demyelinating disease that leads to lesions in the central nervous system. This disease can be tracked and diagnosed using Magnetic Resonance Imaging (MRI). A multitude of multimodality automatic biomedical approaches are used to segment lesions that are not beneficial for patients in terms of cost, time, and usability. The authors of the present paper propose a method employing just one modality (FLAIR image) to segment MS lesions accurately. Methods: A patch-based Convolutional Neural Network (CNN) is designed, inspired by 3D-ResNet and spatial-channel attention module, to segment MS lesions. The proposed method consists of three stages: (1) the Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to the original images and concatenated to the extracted edges to create 4D images; (2) the patches of size [Formula: see text] are randomly selected from the 4D images; and (3) the extracted patches are passed into an attention-based CNN which is used to segment the lesions. Finally, the proposed method was compared to previous studies of the same dataset. Results: The current study evaluates the model with a test set of ISIB challenge data. Experimental results illustrate that the proposed approach significantly surpasses existing methods of Dice similarity and Absolute Volume Difference while the proposed method uses just one modality (FLAIR) to segment the lesions. Conclusion: The authors have introduced an automated approach to segment the lesions, which is based on, at most, two modalities as an input. The proposed architecture comprises convolution, deconvolution, and an SCA-VoxRes module as an attention module. The results show, that the proposed method outperforms well compared to other methods.

Multiple Sclerosis Lesion Analysis in Brain Magnetic Resonance Images: Techniques and Clinical Applications

Multiple sclerosis (MS) is a chronic inflammatory and degenerative disease of the central nervous system, characterized by the appearance of focal lesions in the white and gray matter that topographically correlate with an individual patients neurological symptoms and signs. Magnetic resonance imaging (MRI) provides detailed in-vivo structural information, permitting the quantification and categorization of MS lesions that critically inform disease management. Traditionally, MS lesions have been manually annotated on 2D MRI slices, a process that is inefficient and prone to inter-/intra-observer errors. Recently, automated statistical imaging analysis techniques have been proposed to detect and segment MS lesions based on MRI voxel intensity. However, their effectiveness is limited by the heterogeneity of both MRI data acquisition techniques and the appearance of MS lesions. By learning complex lesion representations directly from images, deep learning techniques have achieved remarkable breakthroughs in the MS lesion segmentation task. Here, we provide a comprehensive review of state-of-the-art automatic statistical and deep-learning MS segmentation methods and discuss current and future clinical applications. Further, we review technical strategies, such as domain adaptation, to enhance MS lesion segmentation in real-world clinical settings.

Deep structured residual encoder-decoder network with a novel loss function for nuclei segmentation of kidney and breast histopathology images

To improve the process of diagnosis and treatment of cancer disease, automatic segmentation of haematoxylin and eosin (H & E) stained cell nuclei from histopathology images is the first step in digital pathology. The proposed deep structured residual encoder-decoder network (DSREDN) focuses on two aspects: first, it effectively utilized residual connections throughout the network and provides a wide and deep encoder-decoder path, which results to capture relevant context and more localized features. Second, vanished boundary of detected nuclei is addressed by proposing an efficient loss function that better train our proposed model and reduces the false prediction which is undesirable especially in healthcare applications. The proposed architecture experimented on three different publicly available H&E stained histopathological datasets namely: (I) Kidney (RCC) (II) Triple Negative Breast Cancer (TNBC) (III) MoNuSeg-2018. We have considered F1-score, Aggregated Jaccard Index (AJI), the total number of parameters, and FLOPs (Floating point operations), which are mostly preferred performance measure metrics for comparison of nuclei segmentation. The evaluated score of nuclei segmentation indicated that the proposed architecture achieved a considerable margin over five state-of-the-art deep learning models on three different histopathology datasets. Visual segmentation results show that the proposed DSREDN model accurately segment the nuclear regions than those of the state-of-the-art methods.

Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation

Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentation are based on either the cross entropy loss, Dice loss or a combination of the two. We propose the Unified Focal loss, a new hierarchical framework that generalises Dice and cross entropy-based losses for handling class imbalance. We evaluate our proposed loss function on five publicly available, class imbalanced medical imaging datasets: CVC-ClinicDB, Digital Retinal Images for Vessel Extraction (DRIVE), Breast Ultrasound 2017 (BUS2017), Brain Tumour Segmentation 2020 (BraTS20) and Kidney Tumour Segmentation 2019 (KiTS19). We compare our loss function performance against six Dice or cross entropy-based loss functions, across 2D binary, 3D binary and 3D multiclass segmentation tasks, demonstrating that our proposed loss function is robust to class imbalance and consistently outperforms the other loss functions. Source code is available at: https://github.com/mlyg/unified-focal-loss.

•

Loss function choice is crucial for class-imbalanced medical imaging datasets.

•

Understanding the relationship between loss functions is key to inform choice.

•

Unified Focal loss generalises Dice and cross-entropy based loss functions.

•

Unified Focal loss outperforms various Dice and cross-entropy based loss functions.

DeepLesionBrain: Towards a broader deep-learning generalization for multiple sclerosis lesion segmentation

Recently, segmentation methods based on Convolutional Neural Networks (CNNs) showed promising performance in automatic Multiple Sclerosis (MS) lesions segmentation. These techniques have even outperformed human experts in controlled evaluation conditions such as Longitudinal MS Lesion Segmentation Challenge (ISBI Challenge). However, state-of-the-art approaches trained to perform well on highly-controlled datasets fail to generalize on clinical data from unseen datasets. Instead of proposing another improvement of the segmentation accuracy, we propose a novel method robust to domain shift and performing well on unseen datasets, called DeepLesionBrain (DLB). This generalization property results from three main contributions. First, DLB is based on a large group of compact 3D CNNs. This spatially distributed strategy aims to produce a robust prediction despite the risk of generalization failure of some individual networks. Second, we propose a hierarchical specialization learning (HSL) by pre-training a generic network over the whole brain, before using its weights as initialization to locally specialized networks. By this end, DLB learns both generic features extracted at global image level and specific features extracted at local image level. Finally, DLB includes a new image quality data augmentation to reduce dependency to training data specificity (e.g., acquisition protocol). DLB generalization was validated in cross-dataset experiments on MSSEG'16, ISBI challenge, and in-house datasets. During experiments, DLB showed higher segmentation accuracy, better segmentation consistency and greater generalization performance compared to state-of-the-art methods. Therefore, DLB offers a robust framework well-suited for clinical practice.

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Purpose:

Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique.

Method:

In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression.

Results:

We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0422$$\end{document}+0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F_1$$\end{document}F1 of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$+ 0.0865$$\end{document}+0.0865 over the baseline.

Conclusion:

The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.

ALL-Net: Anatomical information lesion-wise loss function integrated into neural network for multiple sclerosis lesion segmentation

Accurate detection and segmentation of multiple sclerosis (MS) brain lesions on magnetic resonance images are important for disease diagnosis and treatment. This is a challenging task as lesions vary greatly in size, shape, location, and image contrast. The objective of our study was to develop an algorithm based on deep convolutional neural network integrated with anatomic information and lesion-wise loss function (ALL-Net) for fast and accurate automated segmentation of MS lesions. Distance transformation mapping was used to construct a convolutional module that encoded lesion-specific anatomical information. To overcome the lesion size imbalance during network training and improve the detection of small lesions, a lesion-wise loss function was developed in which individual lesions were modeled as spheres of equal size. On the ISBI-2015 longitudinal MS lesion segmentation challenge dataset (19 subjects in total), ALL-Net achieved an overall score of 93.32 and was amongst the top performing methods. On the larger Cornell MS dataset (176 subjects in total), ALL-Net significantly improved both voxel-wise metrics (Dice improvement of 3.9% to 35.3% with p-values ranging from p < 0.01 to p < 0.0001, and AUC of voxel-wise precision-recall curve improvement of 2.1% to 29.8%) and lesion-wise metrics (lesion-wise F1 score improvement of 12.6% to 29.8% with all p-values p < 0.0001, and AUC of lesion-wise ROC curve improvement of 1.4% to 20.0%) compared to leading publicly available MS lesion segmentation tools.

Emerging deep learning techniques using magnetic resonance imaging data applied in multiple sclerosis and clinical isolated syndrome patients (Review)

Computer-aided diagnosis systems aim to assist clinicians in the early identification of abnormal signs in order to optimize the interpretation of medical images and increase diagnostic precision. Multiple sclerosis (MS) and clinically isolated syndrome (CIS) are chronic inflammatory, demyelinating diseases affecting the central nervous system. Recent advances in deep learning (DL) techniques have led to novel computational paradigms in MS and CIS imaging designed for automatic segmentation and detection of areas of interest and automatic classification of anatomic structures, as well as optimization of neuroimaging protocols. To this end, there are several publications presenting artificial intelligence-based predictive models aiming to increase diagnostic accuracy and to facilitate optimal clinical management in patients diagnosed with MS and/or CIS. The current study presents a thorough review covering DL techniques that have been applied in MS and CIS during recent years, shedding light on their current advances and limitations.

Blood Vessel Segmentation Based on the 3D Residual U-Net

In this paper, we propose blood vessel segmentation based on the 3D residual U-Net method. First, we integrate the residual block structure into the 3D U-Net. By exploring the influence of adding residual blocks at different positions in the 3D U-Net, we establish a novel and effective 3D residual U-Net. In addition, to address the challenges of pixel imbalance, vessel boundary segmentation, and small vessel segmentation, we develop a new weighted Dice loss function with a better effect than the weighted cross-entropy loss function. When training the model, we adopted a two-stage method from coarse-to-fine. In the fine stage, a local segmentation method of 3D sliding window is added. In the model testing phase, we used the 3D fixed-point method. Furthermore, we employ the 3D morphological closed operation to smooth the surfaces of vessels and volume analysis to remove noise blocks. To verify the accuracy and stability of our method, we compare our method with FCN, 3D DenseNet, and 3D U-Net. The experimental results indicate that our method has higher accuracy and better stability than the other studied methods and that the average Dice coefficients for hepatic veins and portal veins reach 71.7% and 76.5% in the coarse stage and 72.5% and 77.2% in the fine stage, respectively. In order to verify the robustness of the model, we conducted the same comparative experiment on the brain vessel datasets, and the average Dice coefficient reached 87.2%.

HD-RDS-UNet: Leveraging Spatial-Temporal Correlation between the Decoder Feature Maps for Lymphoma Segmentation

Lymphoma is a group of malignant tumors originated in the lymphatic system. Automatic and accurate lymphoma segmentation in PET/CT volumes is critical yet challenging in the clinical practice. Recently, UNet-like architectures are widely used for medical image segmentation. The pure UNet-like architectures model the spatial correlation between the feature maps very well, whereas they discard the critical temporal correlation. Some prior work combines UNet with recurrent neural networks (RNNs) to utilize the spatial and temporal correlation simultaneously. However, it is inconvenient to incorporate some advanced techniques for UNet to RNNs, which hampers their further improvements. In this paper, we propose a recurrent dense siamese decoder architecture, which simulates RNNs and can densely utilize the spatial-temporal correlation between the decoder feature maps following a UNet approach. We combine it with a modified hyper dense encoder. Therefore, the proposed model is a UNet with a hyper dense encoder and a recurrent dense siamese decoder (HD-RDS-UNet). To stabilize the training process, we propose a weighted Dice loss with stable gradient and self-adaptive parameters. We perform patient-independent fivefold cross-validation on 3D volumes collected from whole-body PET/CT scans of patients with lymphomas. The experimental results show that the volume-wise average Dice score and sensitivity are 85.58% and 94.63%, respectively. The patient-wise average Dice score and sensitivity are 85.85% and 95.01%, respectively. The different configurations of HD-RDS-UNet consistently show superiority in the performance comparison. Besides, a trained HD-RDS-UNet can be easily pruned, resulting in significantly reduced inference time and memory usage, while keeping very good segmentation performance.

Applications of deep learning techniques for automated multiple sclerosis detection using magnetic resonance imaging: A review

Multiple Sclerosis (MS) is a type of brain disease which causes visual, sensory, and motor problems for people with a detrimental effect on the functioning of the nervous system. In order to diagnose MS, multiple screening methods have been proposed so far; among them, magnetic resonance imaging (MRI) has received considerable attention among physicians. MRI modalities provide physicians with fundamental information about the structure and function of the brain, which is crucial for the rapid diagnosis of MS lesions. Diagnosing MS using MRI is time-consuming, tedious, and prone to manual errors. Research on the implementation of computer aided diagnosis system (CADS) based on artificial intelligence (AI) to diagnose MS involves conventional machine learning and deep learning (DL) methods. In conventional machine learning, feature extraction, feature selection, and classification steps are carried out by using trial and error; on the contrary, these steps in DL are based on deep layers whose values are automatically learn. In this paper, a complete review of automated MS diagnosis methods performed using DL techniques with MRI neuroimaging modalities is provided. Initially, the steps involved in various CADS proposed using MRI modalities and DL techniques for MS diagnosis are investigated. The important preprocessing techniques employed in various works are analyzed. Most of the published papers on MS diagnosis using MRI modalities and DL are presented. The most significant challenges facing and future direction of automated diagnosis of MS using MRI modalities and DL techniques are also provided.

Dynamic-weighting hierarchical segmentation network for medical images

Automatic medical image segmentation plays a crucial role in many medical image analysis applications, such as disease diagnosis and prognosis. Despite the extensive progress of existing deep learning based models for medical image segmentation, they focus on extracting accurate features by designing novel network structures and solely utilize fully connected (FC) layer for pixel-level classification. Considering the insufficient capability of FC layer to encode the extracted diverse feature representations, we propose a Hierarchical Segmentation (HieraSeg) Network for medical image segmentation and devise a Hierarchical Fully Connected (HFC) layer. Specifically, it consists of three classifiers and decouples each category into several subcategories by introducing multiple weight vectors to denote the diverse characteristics in each category. A subcategory-level and a category-level learning schemes are then designed to explicitly enforce the discrepant subcategories and automatically capture the most representative characteristics. Hence, the HFC layer can fit the variant characteristics so as to derive an accurate decision boundary. To enhance the robustness of HieraSeg Network with the variability of lesions, we further propose a Dynamic-Weighting HieraSeg (DW-HieraSeg) Network, which introduces an Image-level Weight Net (IWN) and a Pixel-level Weight Net (PWN) to learn data-driven curriculum. Through progressively incorporating informative images and pixels in an easy-to-hard manner, DW-HieraSeg Network is able to eliminate local optimums and accelerate the training process. Additionally, a class balanced loss is proposed to constrain the PWN for preventing the overfitting problem in minority category. Comprehensive experiments on three benchmark datasets, EndoScene, ISIC and Decathlon, show our newly proposed HieraSeg and DW-HieraSeg Networks achieve state-of-the-art performance, which clearly demonstrates the effectiveness of the proposed approaches for medical image segmentation.

Loss Weightings for Improving Imbalanced Brain Structure Segmentation Using Fully Convolutional Networks

Brain structure segmentation on magnetic resonance (MR) images is important for various clinical applications. It has been automatically performed by using fully convolutional networks. However, it suffers from the class imbalance problem. To address this problem, we investigated how loss weighting strategies work for brain structure segmentation tasks with different class imbalance situations on MR images. In this study, we adopted segmentation tasks of the cerebrum, cerebellum, brainstem, and blood vessels from MR cisternography and angiography images as the target segmentation tasks. We used a U-net architecture with cross-entropy and Dice loss functions as a baseline and evaluated the effect of the following loss weighting strategies: inverse frequency weighting, median inverse frequency weighting, focal weighting, distance map-based weighting, and distance penalty term-based weighting. In the experiments, the Dice loss function with focal weighting showed the best performance and had a high average Dice score of 92.8% in the binary-class segmentation tasks, while the cross-entropy loss functions with distance map-based weighting achieved the Dice score of up to 93.1% in the multi-class segmentation tasks. The results suggested that the distance map-based and the focal weightings could boost the performance of cross-entropy and Dice loss functions in class imbalanced segmentation tasks, respectively.

icobrain ms 5.1: Combining unsupervised and supervised approaches for improving the detection of multiple sclerosis lesions

Multiple sclerosis (MS) is a chronic autoimmune, inflammatory neurological disease of the central nervous system. Its diagnosis nowadays commonly includes performing an MRI scan, as it is the most sensitive imaging test for MS. MS plaques are commonly identified from fluid-attenuated inversion recovery (FLAIR) images as hyperintense regions that are highly varying in terms of their shapes, sizes and locations, and are routinely classified in accordance to the McDonald criteria. Recent years have seen an increase in works that aimed at development of various semi-automatic and automatic methods for detection, segmentation and classification of MS plaques. In this paper, we present an automatic combined method, based on two pipelines: a traditional unsupervised machine learning technique and a deep-learning attention-gate 3D U-net network. The deep-learning network is specifically trained to address the weaker points of the traditional approach, namely difficulties in segmenting infratentorial and juxtacortical plaques in real-world clinical MRIs. It was trained and validated on a multi-center multi-scanner dataset that contains 159 cases, each with T1 weighted (T1w) and FLAIR images, as well as manual delineations of the MS plaques, segmented and validated by a panel of raters. The detection rate was quantified using lesion-wise Dice score. A simple label fusion is implemented to combine the output segmentations of the two pipelines. This combined method improves the detection of infratentorial and juxtacortical lesions by 14% and 31% respectively, in comparison to the unsupervised machine learning pipeline that was used as a performance assessment baseline.

Radius-optimized efficient template matching for lesion detection from brain images

Computer-aided detection of brain lesions from volumetric magnetic resonance imaging (MRI) is in demand for fast and automatic diagnosis of neural diseases. The template-matching technique can provide satisfactory outcome for automatic localization of brain lesions; however, finding the optimal template size that maximizes similarity of the template and the lesion remains challenging. This increases the complexity of the algorithm and the requirement for computational resources, while processing large MRI volumes with three-dimensional (3D) templates. Hence, reducing the computational complexity of template matching is needed. In this paper, we first propose a mathematical framework for computing the normalized cross-correlation coefficient (NCCC) as the similarity measure between the MRI volume and approximated 3D Gaussian template with linear time complexity, [Formula: see text], as opposed to the conventional fast Fourier transform (FFT) based approach with the complexity [Formula: see text], where [Formula: see text] is the number of voxels in the image and [Formula: see text] is the number of tried template radii. We then propose a mathematical formulation to analytically estimate the optimal template radius for each voxel in the image and compute the NCCC with the location-dependent optimal radius, reducing the complexity to [Formula: see text]. We test our methods on one synthetic and two real multiple-sclerosis databases, and compare their performances in lesion detection with FFT and a state-of-the-art lesion prediction algorithm. We demonstrate through our experiments the efficiency of the proposed methods for brain lesion detection and their comparable performance with existing techniques.