Joint Optic Disc and Cup Segmentation Based on Multi-Label Deep Network and Polar Transformation

Beyond the eye: A relational model for early dementia detection using retinal OCTA images

Early detection of dementia, such as Alzheimer's disease (AD) or mild cognitive impairment (MCI), is essential to enable timely intervention and potential treatment. Accurate detection of AD/MCI is challenging due to the high complexity, cost, and often invasive nature of current diagnostic techniques, which limit their suitability for large-scale population screening. Given the shared embryological origins and physiological characteristics of the retina and brain, retinal imaging is emerging as a potentially rapid and cost-effective alternative for the identification of individuals with or at high risk of AD. In this paper, we present a novel PolarNet+ that uses retinal optical coherence tomography angiography (OCTA) to discriminate early-onset AD (EOAD) and MCI subjects from controls. Our method first maps OCTA images from Cartesian coordinates to polar coordinates, allowing approximate sub-region calculation to implement the clinician-friendly early treatment of diabetic retinopathy study (ETDRS) grid analysis. We then introduce a multi-view module to serialize and analyze the images along three dimensions for comprehensive, clinically useful information extraction. Finally, we abstract the sequence embedding into a graph, transforming the detection task into a general graph classification problem. A regional relationship module is applied after the multi-view module to explore the relationship between the sub-regions. Such regional relationship analyses validate known eye-brain links and reveal new discriminative patterns. The proposed model is trained, tested, and validated on four retinal OCTA datasets, including 1,671 participants with AD, MCI, and healthy controls. Experimental results demonstrate the performance of our model in detecting AD and MCI with an AUC of 88.69% and 88.02%, respectively. Our results provide evidence that retinal OCTA imaging, coupled with artificial intelligence, may serve as a rapid and non-invasive approach for large-scale screening of AD and MCI. The code is available at https://github.com/iMED-Lab/PolarNet-Plus-PyTorch, and the dataset is also available upon request.

Multidimensional Directionality-Enhanced Segmentation via large vision model

Optical Coherence Tomography (OCT) facilitates a comprehensive examination of macular edema and associated lesions. Manual delineation of retinal fluid is labor-intensive and error-prone, necessitating an automated diagnostic and therapeutic planning mechanism. Conventional supervised learning models are hindered by dataset limitations, while Transformer-based large vision models exhibit challenges in medical image segmentation, particularly in detecting small, subtle lesions in OCT images. This paper introduces the Multidimensional Directionality-Enhanced Retinal Fluid Segmentation framework (MD-DERFS), which reduces the limitations inherent in conventional supervised models by adapting a transformer-based large vision model for macular edema segmentation. The proposed MD-DERFS introduces a Multi-Dimensional Feature Re-Encoder Unit (MFU) to augment the model's proficiency in recognizing specific textures and pathological features through directional prior extraction and an Edema Texture Mapping Unit (ETMU), a Cross-scale Directional Insight Network (CDIN) furnishes a holistic perspective spanning local to global details, mitigating the large vision model's deficiencies in capturing localized feature information. Additionally, the framework is augmented by a Harmonic Minutiae Segmentation Equilibrium loss (LHMSE) that can address the challenges of data imbalance and annotation scarcity in macular edema datasets. Empirical validation on the MacuScan-8k dataset shows that MD-DERFS surpasses existing segmentation methodologies, demonstrating its efficacy in adapting large vision models for boundary-sensitive medical imaging tasks. The code is publicly available at https://github.com/IMOP-lab/MD-DERFS-Pytorch.git.

Optimizing deep learning models for glaucoma screening with vision transformers for resource efficiency and the pie augmentation method

Glaucoma is the leading cause of irreversible vision impairment, emphasizing the critical need for early detection. Typically, AI-based glaucoma screening relies on fundus imaging. To tackle the resource and time challenges in glaucoma screening with convolutional neural network (CNN), we chose the Data-efficient image Transformers (DeiT), a vision transformer, known for its reduced computational demands, with preprocessing time decreased by a factor of 10. Our approach utilized the meticulously annotated GlauCUTU-DATA dataset, curated by ophthalmologists through consensus, encompassing both unanimous agreement (3/3) and majority agreement (2/3) data. However, DeiT's performance was initially lower than CNN. Therefore, we introduced the "pie method," an augmentation method aligned with the ISNT rule. Along with employing polar transformation to improved cup region visibility and alignment with the vision transformer's input to elevated performance levels. The classification results demonstrated improvements comparable to CNN. Using the 3/3 data, excluding the superior and nasal regions, especially in glaucoma suspects, sensitivity increased by 40.18% from 47.06% to 88.24%. The average area under the curve (AUC) ± standard deviation (SD) for glaucoma, glaucoma suspects, and no glaucoma were 92.63 ± 4.39%, 92.35 ± 4.39%, and 92.32 ± 1.45%, respectively. With the 2/3 data, excluding the superior and temporal regions, sensitivity for diagnosing glaucoma increased by 11.36% from 47.73% to 59.09%. The average AUC ± SD for glaucoma, glaucoma suspects, and no glaucoma were 68.22 ± 4.45%, 68.23 ± 4.39%, and 73.09 ± 3.05%, respectively. For both datasets, the AUC values for glaucoma, glaucoma suspects, and no glaucoma were 84.53%, 84.54%, and 91.05%, respectively, which approach the performance of a CNN model that achieved 84.70%, 84.69%, and 93.19%, respectively. Moreover, the incorporation of attention maps from DeiT facilitated the precise localization of clinically significant areas, such as the disc rim and notching, thereby enhancing the overall effectiveness of glaucoma screening.

Randomness-Restricted Diffusion Model for Ocular Surface Structure Segmentation

Ocular surface diseases affect a significant portion of the population worldwide. Accurate segmentation and quantification of different ocular surface structures are crucial for the understanding of these diseases and clinical decision-making. However, the automated segmentation of the ocular surface structure is relatively unexplored and faces several challenges. Ocular surface structure boundaries are often inconspicuous and obscured by glare from reflections. In addition, the segmentation of different ocular structures always requires training of multiple individual models. Thus, developing a one-model-fits-all segmentation approach is desirable. In this paper, we introduce a randomness-restricted diffusion model for multiple ocular surface structure segmentation. First, a time-controlled fusion-attention module (TFM) is proposed to dynamically adjust the information flow within the diffusion model, based on the temporal relationships between the network's input and time. TFM enables the network to effectively utilize image features to constrain the randomness of the generation process. We further propose a low-frequency consistency filter and a new loss to alleviate model uncertainty and error accumulation caused by the multi-step denoising process. Extensive experiments have shown that our approach can segment seven different ocular surface structures. Our method performs better than both dedicated ocular surface segmentation methods and general medical image segmentation methods. We further validated the proposed method over two clinical datasets, and the results demonstrated that it is beneficial to clinical applications, such as the meibomian gland dysfunction grading and aqueous deficient dry eye diagnosis.

MEFA-Net: A mask enhanced feature aggregation network for polyp segmentation

Accurate polyp segmentation is crucial for early diagnosis and treatment of colorectal cancer. This is a challenging task for three main reasons: (i) the problem of model overfitting and weak generalization due to the multi-center distribution of data; (ii) the problem of interclass ambiguity caused by motion blur and overexposure to endoscopic light; and (iii) the problem of intraclass inconsistency caused by the variety of morphologies and sizes of the same type of polyps. To address these challenges, we propose a new high-precision polyp segmentation framework, MEFA-Net, which consists of three modules, including the plug-and-play Mask Enhancement Module (MEG), Separable Path Attention Enhancement Module (SPAE), and Dynamic Global Attention Pool Module (DGAP). Specifically, firstly, the MEG module regionally masks the high-energy regions of the environment and polyps through a mask, which guides the model to rely on only a small amount of information to distinguish between polyps and background features, avoiding the model from overfitting the environmental information, and improving the robustness of the model. At the same time, this module can effectively counteract the "dark corner phenomenon" in the dataset and further improve the generalization performance of the model. Next, the SPAE module can effectively alleviate the inter-class fuzzy problem by strengthening the feature expression. Then, the DGAP module solves the intra-class inconsistency problem by extracting the invariance of scale, shape and position. Finally, we propose a new evaluation metric, MultiColoScore, for comprehensively evaluating the segmentation performance of the model on five datasets with different domains. We evaluated the new method quantitatively and qualitatively on five datasets using four metrics. Experimental results show that MEFA-Net significantly improves the accuracy of polyp segmentation and outperforms current state-of-the-art algorithms. Code posted on https://github.com/847001315/MEFA-Net.

Rim learning framework based on TS-GAN: A new paradigm of automated glaucoma screening from fundus images

Glaucoma detection from fundus images often relies on biomarkers such as the Cup-to-Disc Ratio (CDR) and Rim-to-Disc Ratio (RDR). However, precise segmentation of the optic cup and disc is challenging due to low-contrast boundaries and the interference of blood vessels and optic nerves. This article presents a novel automated framework for glaucoma detection that focuses on the rim structure as a biomarker, excluding the conventional reliance on CDR and RDR. The proposed framework introduces a Teacher-Student Generative Adversarial Network (TS-GAN) for precise segmentation of the optic cup and disc, along with a SqueezeNet for glaucoma detection. The Teacher model uses an attention-based CNN encoder-decoder, while the Student model incorporates Expectation Maximization to enhance segmentation performance. By combining implicit generative modeling and explicit probability density modeling, the TS-GAN effectively addresses the mode collapse problem seen in existing GANs. A rim generator processes the segmented cup and disc to produce the rim, which serves as input to SqueezeNet for glaucoma classification. The framework has been extensively tested on diverse fundus image datasets, including a private dataset, demonstrating superior segmentation and detection accuracy compared to state-of-the-art models. Results show its effectiveness in early glaucoma detection, offering higher accuracy and reliability. This innovative framework provides a robust tool for ophthalmologists, enabling efficient glaucoma management and reducing the risk of vision loss.

Automatic Segmentation of Bone Graft in Maxillary Sinus via Distance Constrained Network Guided by Prior Anatomical Knowledge

Maxillary Sinus Lifting is a crucial surgical procedure for addressing insufficient alveolar bone mass andsevere resorption in dental implant therapy. To accurately analyze the geometry changesof the bone graft (BG) in the maxillary sinus (MS), it is essential to perform quantitative analysis. However, automated BG segmentation remains a major challenge due to the complex local appearance, including blurred boundaries, lesion interference, implant and artifact interference, and BG exceeding the MS. Currently, there are few tools available that can efficiently and accurately segment BG from cone beam computed tomography (CBCT) image. In this paper, we propose a distance-constrained attention network guided by prior anatomical knowledge for the automatic segmentation of BG. First, a guidance strategy of preoperative prior anatomical knowledge is added to a deep neural network (DNN), which improves its ability to identify the dividing line between the MS and BG. Next, a coordinate attention gate is proposed, which utilizes the synergy of channel and position attention to highlight salient features from the skip connections. Additionally, the geodesic distance constraint is introduced into the DNN to form multi-task predictions, which reduces the deviation of the segmentation result. In the test experiment, the proposed DNN achieved a Dice similarity coefficient of 85.48 6.38%, an average surface distance error is 0.57 0.34mm, and a 95% Hausdorff distance of 2.64 2.09mm, which is superior to the comparison networks. It markedly improves the segmentation accuracy and efficiency of BG and has potential applications in analyzing its volume change and absorption rate in the future.

Heatmap-Based Active Shape Model for Landmark Detection in Lumbar X-ray Images

Medical staff inspect lumbar X-ray images to diagnose lumbar spine diseases, and the analysis process is currently automated using deep-learning techniques. The detection of landmarks is necessary in the automatic process of localizing the position and identifying the morphological features of the vertebrae. However, detection errors may occur owing to the noise and ambiguity of images, as well as individual variations in the shape of the lumbar vertebrae. This study proposes a method to improve the robustness of landmark detection results. This method assumes that landmarks are detected by a convolutional neural network-based two-step model consisting of Pose-Net and M-Net. The model generates a heatmap response to indicate the probable landmark positions. The proposed method then corrects the landmark positions using the heatmap response and active shape model, which employs statistical information on the landmark distribution. Experiments were conducted using 3600 lumbar X-ray images, and the results showed that the landmark detection error was reduced by the proposed method. The average value of maximum errors decreased by 5.58% after applying the proposed method, which combines the outstanding image analysis capabilities of deep learning with statistical shape constraints on landmark distribution. The proposed method could also be easily integrated with other techniques to increase the robustness of landmark detection results such as CoordConv layers and non-directional part affinity field. This resulted in a further enhancement in the landmark detection performance. These advantages can improve the reliability of automatic systems used to inspect lumbar X-ray images. This will benefit both patients and medical staff by reducing medical expenses and increasing diagnostic efficiency.

MIU-Net: Advanced multi-scale feature extraction and imbalance mitigation for optic disc segmentation

Pathological myopia is a severe eye condition that can cause serious complications like retinal detachment and macular degeneration, posing a threat to vision. Optic disc segmentation helps measure changes in the optic disc and observe the surrounding retina, aiding early detection of pathological myopia. However, these changes make segmentation difficult, resulting in accuracy levels that are not suitable for clinical use. To address this, we propose a new model called MIU-Net, which improves segmentation performance through several innovations. First, we introduce a multi-scale feature extraction (MFE) module to capture features at different scales, helping the model better identify optic disc boundaries in complex images. Second, we design a dual attention module that combines channel and spatial attention to focus on important features and improve feature use. To tackle the imbalance between optic disc and background pixels, we use focal loss to enhance the model's ability to detect minority optic disc pixels. We also apply data augmentation techniques to increase data diversity and address the lack of training data. Our model was tested on the iChallenge-PM and iChallenge-AMD datasets, showing clear improvements in accuracy and robustness compared to existing methods. The experimental results demonstrate the effectiveness and potential of our model in diagnosing pathological myopia and other medical image processing tasks.

Multi scale multi attention network for blood vessel segmentation in fundus images

Precise segmentation of retinal vasculature is crucial for the early detection, diagnosis, and treatment of vision-threatening ailments. However, this task is challenging due to limited contextual information, variations in vessel thicknesses, the complexity of vessel structures, and the potential for confusion with lesions. In this paper, we introduce a novel approach, the MSMA Net model, which overcomes these challenges by replacing traditional convolution blocks and skip connections with an improved multi-scale squeeze and excitation block (MSSE Block) and Bottleneck residual paths (B-Res paths) with spatial attention blocks (SAB). Our experimental findings on publicly available datasets of fundus images, specifically DRIVE, STARE, CHASE_DB1, HRF and DR HAGIS consistently demonstrate that our approach outperforms other segmentation techniques, achieving higher accuracy, sensitivity, Dice score, and area under the receiver operator characteristic (AUC) in the segmentation of blood vessels with different thicknesses, even in situations involving diverse contextual information, the presence of coexisting lesions, and intricate vessel morphologies.

C 2 MAL: cascaded network-guided class-balanced multi-prototype auxiliary learning for source-free domain adaptive medical image segmentation

Source-free domain adaptation (SFDA) has become crucial in medical image analysis, enabling the adaptation of source models across diverse datasets without labeled target domain images. Self-training, a popular SFDA approach, iteratively refines self-generated pseudo-labels using unlabeled target domain data to adapt a pre-trained model from the source domain. However, it often faces model instability due to incorrect pseudo-label accumulation and foreground-background class imbalance. This paper presents a pioneering SFDA framework, named cascaded network-guided class-balanced multi-prototype auxiliary learning (C2 MAL), to enhance model stability. Firstly, we introduce the cascaded translation-segmentation network (CTS-Net), which employs iterative learning between translation and segmentation networks to generate accurate pseudo-labels. The CTS-Net employs a translation network to synthesize target-like images from unreliable predictions of the initial target domain images. The synthesized results refine segmentation network training, ensuring semantic alignment and minimizing visual disparities. Subsequently, reliable pseudo-labels guide the class-balanced multi-prototype auxiliary learning network (CMAL-Net) for effective model adaptation. CMAL-Net incorporates a new multi-prototype auxiliary learning strategy with a memory network to complement source domain data. We propose a class-balanced calibration loss and multi-prototype-guided symmetry cross-entropy loss to tackle class imbalance issue and enhance model adaptability to the target domain. Extensive experiments on four benchmark fundus image datasets validate the superiority of C2 MAL over state-of-the-art methods, especially in scenarios with significant domain shifts. Our code is available at https://github.com/yxk-art/C2MAL .

Enhancing stereotactic ablative boost radiotherapy dose prediction for bulky lung cancer: A multi-scale dilated network approach with scale-balanced structure loss

PURPOSE

Partial stereotactic ablative boost radiotherapy (P-SABR) effectively treats bulky lung cancer; however, the planning process for P-SABR requires repeated dose calculations. To improve planning efficiency, we proposed a novel deep learning method that utilizes limited data to accurately predict the three-dimensional (3D) dose distribution of the P-SABR plan for bulky lung cancer.

METHODS

We utilized data on 74 patients diagnosed with bulky lung cancer who received P-SABR treatment. The patient dataset was randomly divided into a training set (51 plans) with augmentation, validation set (7 plans), and testing set (16 plans). We devised a 3D multi-scale dilated network (MD-Net) and integrated a scale-balanced structure loss into the loss function. A comparative analysis with a classical network and other advanced networks with multi-scale analysis capabilities and other loss functions was conducted based on the dose distributions in terms of the axial view, average dose scores (ADSs), and average absolute differences of dosimetric indices (AADDIs). Finally, we analyzed the predicted dosimetric indices against the ground-truth values and compared the predicted dose-volume histogram (DVH) with the ground-truth DVH.

RESULTS

Our proposed dose prediction method for P-SABR plans for bulky lung cancer demonstrated strong performance, exhibiting a significant improvement in predicting multiple indicators of regions of interest (ROIs), particularly the gross target volume (GTV). Our network demonstrated increased accuracy in most dosimetric indices and dose scores in different ROIs. The proposed loss function significantly enhanced the predictive performance of the dosimetric indices. The predicted dosimetric indices and DVHs were equivalent to the ground-truth values.

CONCLUSION

Our study presents an effective model based on limited datasets, and it exhibits high accuracy in the dose prediction of P-SABR plans for bulky lung cancer. This method has potential as an automated tool for P-SABR planning and can help optimize treatments and improve planning efficiency.

Medical Image Segmentation Review: The Success of U-Net

Automatic medical image segmentation is a crucial topic in the medical domain and successively a critical counterpart in the computer-aided diagnosis paradigm. U-Net is the most widespread image segmentation architecture due to its flexibility, optimized modular design, and success in all medical image modalities. Over the years, the U-Net model has received tremendous attention from academic and industrial researchers who have extended it to address the scale and complexity created by medical tasks. These extensions are commonly related to enhancing the U-Net's backbone, bottleneck, or skip connections, or including representation learning, or combining it with a Transformer architecture, or even addressing probabilistic prediction of the segmentation map. Having a compendium of different previously proposed U-Net variants makes it easier for machine learning researchers to identify relevant research questions and understand the challenges of the biological tasks that challenge the model. In this work, we discuss the practical aspects of the U-Net model and organize each variant model into a taxonomy. Moreover, to measure the performance of these strategies in a clinical application, we propose fair evaluations of some unique and famous designs on well-known datasets. Furthermore, we provide a comprehensive implementation library with trained models. In addition, for ease of future studies, we created an online list of U-Net papers with their possible official implementation.

Research trends and hotspots in fundus image segmentation from 2007 to 2023: A bibliometric analysis

Objective

To understand the current status, research hotspots, and trends of automatic segmentation of fundus lesion images worldwide, providing a reference for subsequent related studies.

Methods

The electronic database Web of Science Core Collection was searched for research in the field of automatic segmentation of fundus lesion images from 2007 to 2023. Visualization maps of countries, authors, institutions, journals, references, and keywords were generated and analyzed using the CiteSpace and VOSviewer software.

Results

After deduplication, 707 publications were sorted out, showing an overall increasing trend in publication volume. The countries with the highest publication counts were China, followed by India, the USA, the UK, Spain, Pakistan, and Singapore. A high degree of collaboration was observed among authors, and they cooperated widely. The keywords included "diabetic retinopathy," "deep learning," "vessel segmentation," "retinal images," "optic disc localization," and so forth, with keyword bursts starting in 2018 for "retinal images," "machine learning," "biomedical imaging," "deep learning," "convolutional neural networks," and "transfer learning." The most prolific author was U Rajendra Acharya from the University of Southern Queensland, and the journal with the most publications was Computer Methods and Programs in Biomedicine.

Conclusions

Compared with manual segmentation of fundus lesion images, the use of deep learning models for segmentation is more efficient and accurate, which is crucial for patients with eye diseases. Although the number of related publications globally is relatively small, a growing trend is still witnessed, with broad connections between countries and authors, mainly concentrated in East Asia and Europe. Research institutions in this field are limited, and hence, the research on diabetic retinopathy and retinal vessel segmentation should be strengthened to promote the development of this area.

Development of a Cost-Efficient and Glaucoma-Specialized OD/OC Segmentation Model for Varying Clinical Scenarios

Most existing optic disc (OD) and cup (OC) segmentation models are biased to the dominant size and easy class (normal class), resulting in suboptimal performances on glaucoma-confirmed samples. Thus, these models are not optimal choices for assisting in tracking glaucoma progression and prognosis. Moreover, fully supervised models employing annotated glaucoma samples can achieve superior performances, although restricted by the high cost of collecting and annotating the glaucoma samples. Therefore, in this paper, we are dedicated to developing a glaucoma-specialized model by exploiting low-cost annotated normal fundus images, simultaneously adapting various common scenarios in clinical practice. We employ a contrastive learning and domain adaptation-based model by exploiting shared knowledge from normal samples. To capture glaucoma-related features, we utilize a Gram matrix to encode style information and the domain adaptation strategy to encode domain information, followed by narrowing the style and domain gaps between normal and glaucoma samples by contrastive and adversarial learning, respectively. To validate the efficacy of our proposed model, we conducted experiments utilizing two public datasets to mimic various common scenarios. The results demonstrate the superior performance of our proposed model across multi-scenarios, showcasing its proficiency in both the segmentation- and glaucoma-related metrics. In summary, our study illustrates a concerted effort to target confirmed glaucoma samples, mitigating the inherent bias issue in most existing models. Moreover, we propose an annotation-efficient strategy that exploits low-cost, normal-labeled fundus samples, mitigating the economic- and labor-related burdens by employing a fully supervised strategy. Simultaneously, our approach demonstrates its adaptability across various scenarios, highlighting its potential utility in both assisting in the monitoring of glaucoma progression and assessing glaucoma prognosis.

Detection of optic disc in human retinal images utilizing the Bitterling Fish Optimization (BFO) algorithm

Early detection and correct identification of the optic disc (OD) on scanned retinal images are significant for diagnosing and treating several ophthalmic conditions, including glaucoma and diabetic retinopathy. Conventional methods for detecting the OD often struggle with processing retinal images due to noise, changes in illumination, and complex overlapping images. This study presents the development of effective and accurate fixation of the optic disc using the Bitterling Fish Optimization (BFO) algorithm, which enhances the processes of OD imaging in speed and precision. The proposed method begins with image enhancement and noise suppression for preprocessing, followed by applying the BFO algorithm to locate and delineate the OD region. The performance evaluation of the algorithm was conducted within several public domain retinal images, including DRIVE, STARE, ORIGA, DRISHTI-GS, DiaRetDB0, and DiaRetDB1 datasets about some internal metrics: sensitivity (SE), specificity (SP), accuracy (ACC), DICE overlap coefficient, overlap and time of processing respectively. The technique based on BFO provided better results, with 99.33%, 99.94%, and 98.22% accuracy achieved for OD in DRIVE, DRISHTI-GS, and DiaRetDB 1, respectively. The approach also demonstrated high overlaps and good DICE results, with a DICE coefficient of 0.9501 for the DRISHTI-GS database. On average, the processing time per image was less than 2.5 s, proving the approach's efficiency in computations. The BFO approach has demonstrated its effectiveness and scalability in detecting optic discs in retinal images in an automated manner. It showed impressive performance levels in terms of computation time and accuracy and was variation resistant irrespective of the quality of the image and the pathology present on it. This method holds significant potential for clinical use, providing a meaningful way of diagnosing and managing ocular disease at an early stage.

Redefining retinal vessel segmentation: empowering advanced fundus image analysis with the potential of GANs

Retinal vessel segmentation is a critical task in fundus image analysis, providing essential insights for diagnosing various retinal diseases. In recent years, deep learning (DL) techniques, particularly Generative Adversarial Networks (GANs), have garnered significant attention for their potential to enhance medical image analysis. This paper presents a novel approach for retinal vessel segmentation by harnessing the capabilities of GANs. Our method, termed GANVesselNet, employs a specialized GAN architecture tailored to the intricacies of retinal vessel structures. In GANVesselNet, a dual-path network architecture is employed, featuring an Auto Encoder-Decoder (AED) pathway and a UNet-inspired pathway. This unique combination enables the network to efficiently capture multi-scale contextual information, improving the accuracy of vessel segmentation. Through extensive experimentation on publicly available retinal datasets, including STARE and DRIVE, GANVesselNet demonstrates remarkable performance compared to traditional methods and state-of-the-art deep learning approaches. The proposed GANVesselNet exhibits superior sensitivity (0.8174), specificity (0.9862), and accuracy (0.9827) in segmenting retinal vessels on the STARE dataset, and achieves commendable results on the DRIVE dataset with sensitivity (0.7834), specificity (0.9846), and accuracy (0.9709). Notably, GANVesselNet achieves remarkable performance on previously unseen data, underscoring its potential for real-world clinical applications. Furthermore, we present qualitative visualizations of the generated vessel segmentations, illustrating the network's proficiency in accurately delineating retinal vessels. In summary, this paper introduces GANVesselNet, a novel and powerful approach for retinal vessel segmentation. By capitalizing on the advanced capabilities of GANs and incorporating a tailored network architecture, GANVesselNet offers a quantum leap in retinal vessel segmentation accuracy, opening new avenues for enhanced fundus image analysis and improved clinical decision-making.

Lightweight Optic Disc and Optic Cup Segmentation Based on MobileNetv3 Convolutional Neural Network

Glaucoma represents a significant global contributor to blindness. Accurately segmenting the optic disc (OD) and optic cup (OC) to obtain precise CDR is essential for effective screening. However, existing convolutional neural network (CNN)-based segmentation techniques are often limited by high computational demands and long inference times. This paper proposes an efficient end-to-end method for OD and OC segmentation, utilizing the lightweight MobileNetv3 network as the core feature-extraction module. Our approach combines boundary branches with adversarial learning, to achieve multi-label segmentation of the OD and OC. We validated our proposed approach across three public available datasets: Drishti-GS, RIM-ONE-r3, and REFUGE. The outcomes reveal that the Dice coefficients for the segmentation of OD and OC within these datasets are 0.974/0.900, 0.966/0.875, and 0.962/0.880, respectively. Additionally, our method substantially lowers computational complexity and inference time, thereby enabling efficient and precise segmentation of the optic disc and optic cup.

Smart Vision Transparency: Efficient Ocular Disease Prediction Model Using Explainable Artificial Intelligence

The early prediction of ocular disease is certainly an obligatory concern in the domain of ophthalmic medicine. Although modern scientific discoveries have shown the potential to treat eye diseases by using artificial intelligence (AI) and machine learning, explainable AI remains a crucial challenge confronting this area of research. Although some traditional methods put in significant effort, they cannot accurately predict the proper ocular diseases. However, incorporating AI into diagnosing eye diseases in healthcare complicates the situation as the decision-making process of AI demonstrates complexity, which is a significant concern, especially in major sectors like ocular disease prediction. The lack of transparency in the AI models may hinder the confidence and trust of the doctors and the patients, as well as their perception of the AI and its abilities. Accordingly, explainable AI is significant in ensuring trust in the technology, enhancing clinical decision-making ability, and deploying ocular disease detection. This research proposed an efficient transfer learning model for eye disease prediction to transform smart vision potential in the healthcare sector and meet conventional approaches' challenges while integrating explainable artificial intelligence (XAI). The integration of XAI in the proposed model ensures the transparency of the decision-making process through the comprehensive provision of rationale. This proposed model provides promising results with 95.74% accuracy and explains the transformative potential of XAI in advancing ocular healthcare. This significant milestone underscores the effectiveness of the proposed model in accurately determining various types of ocular disease. It is clearly shown that the proposed model is performing better than the previously published methods.

Multi-scale input layers and dense decoder aggregation network for COVID-19 lesion segmentation from CT scans

Accurate segmentation of COVID-19 lesions from medical images is essential for achieving precise diagnosis and developing effective treatment strategies. Unfortunately, this task presents significant challenges, owing to the complex and diverse characteristics of opaque areas, subtle differences between infected and healthy tissue, and the presence of noise in CT images. To address these difficulties, this paper designs a new deep-learning architecture (named MD-Net) based on multi-scale input layers and dense decoder aggregation network for COVID-19 lesion segmentation. In our framework, the U-shaped structure serves as the cornerstone to facilitate complex hierarchical representations essential for accurate segmentation. Then, by introducing the multi-scale input layers (MIL), the network can effectively analyze both fine-grained details and contextual information in the original image. Furthermore, we introduce an SE-Conv module in the encoder network, which can enhance the ability to identify relevant information while simultaneously suppressing the transmission of extraneous or non-lesion information. Additionally, we design a dense decoder aggregation (DDA) module to integrate feature distributions and important COVID-19 lesion information from adjacent encoder layers. Finally, we conducted a comprehensive quantitative analysis and comparison between two publicly available datasets, namely Vid-QU-EX and QaTa-COV19-v2, to assess the robustness and versatility of MD-Net in segmenting COVID-19 lesions. The experimental results show that the proposed MD-Net has superior performance compared to its competitors, and it exhibits higher scores on the Dice value, Matthews correlation coefficient (Mcc), and Jaccard index. In addition, we also conducted ablation studies on the Vid-QU-EX dataset to evaluate the contributions of each key component within the proposed architecture.

Segmentation of coronary artery based on discriminative frequency learning and coronary-geometric refinement

Coronary artery segmentation is crucial for physicians to identify and locate plaques and stenosis using coronary computed tomography angiography (CCTA). However, the low contrast of CCTA images and the intricate structures of coronary arteries make this task challenging. To address these difficulties, we propose a novel model, the DFS-PDS network. This network comprises two subnetworks: a discriminative frequency segment subnetwork (DFS) and a position domain scales subnetwork (PDS). DFS introduced a gated mechanism within the feed-forward network, leveraging the Joint Photographic Experts Group (JPEG) compression algorithm, to discriminatively determine which low- and high-frequency information of the features should be preserved for latent image segmentation. The PDS aims to learn the shape prototype by predicting the radius. Additionally, our model has the consistent ability to guarantee region and boundary features through boundary consistency loss. During training, both subnetworks are optimized jointly, and in the testing stage, the coarse segmentation and radius prediction are produced. A coronary-geometric refinement method refines the segmentation masks by leveraging the shape prior to being reconstructed from the radius map, reducing the difficulty of segmenting coronary artery structures from complex surrounding structures. The DFS-PDS network is compared with state-of-the-art (SOTA) methods on two coronary artery datasets to evaluate its performance. The experimental results demonstrate that the DFS-PDS network performs better than the SOTA models, including Vnet, nnUnet, DDT, CS2-Net, Unetr, and CAS-Net, in terms of Dice or connectivity evaluation metrics.

Accurate segmentation of intracellular organelle networks using low-level features and topological self-similarity

MOTIVATION

Intracellular organelle networks (IONs) such as the endoplasmic reticulum (ER) network and the mitochondrial (MITO) network serve crucial physiological functions. The morphology of these networks plays a critical role in mediating their functions. Accurate image segmentation is required for analyzing the morphology and topology of these networks for applications such as molecular mechanism analysis and drug target screening. So far, however, progress has been hindered by their structural complexity and density.

RESULTS

In this study, we first establish a rigorous performance baseline for accurate segmentation of these organelle networks from fluorescence microscopy images by optimizing a baseline U-Net model. We then develop the multi-resolution encoder (MRE) and the hierarchical fusion loss (Lhf) based on two inductive components, namely low-level features and topological self-similarity, to assist the model in better adapting to the task of segmenting IONs. Empowered by MRE and Lhf, both U-Net and Pyramid Vision Transformer (PVT) outperform competing state-of-the-art models such as U-Net++, HR-Net, nnU-Net, and TransUNet on custom datasets of the ER network and the MITO network, as well as on public datasets of another biological network, the retinal blood vessel network. In addition, integrating MRE and Lhf with models such as HR-Net and TransUNet also enhances their segmentation performance. These experimental results confirm the generalization capability and potential of our approach. Furthermore, accurate segmentation of the ER network enables analysis that provides novel insights into its dynamic morphological and topological properties.

AVAILABILITY AND IMPLEMENTATION

Code and data are openly accessible at https://github.com/cbmi-group/MRE.

Unsupervised Domain Adaptation Fundus Image Segmentation via Multi-Scale Adaptive Adversarial Learning

Segmentation of the Optic Disc (OD) and Optic Cup (OC) is crucial for the early detection and treatment of glaucoma. Despite the strides made in deep neural networks, incorporating trained segmentation models for clinical application remains challenging due to domain shifts arising from disparities in fundus images across different healthcare institutions. To tackle this challenge, this study introduces an innovative unsupervised domain adaptation technique called Multi-scale Adaptive Adversarial Learning (MAAL), which consists of three key components. The Multi-scale Wasserstein Patch Discriminator (MWPD) module is designed to extract domain-specific features at multiple scales, enhancing domain classification performance and offering valuable guidance for the segmentation network. To further enhance model generalizability and explore domain-invariant features, we introduce the Adaptive Weighted Domain Constraint (AWDC) module. During training, this module dynamically assigns varying weights to different scales, allowing the model to adaptively focus on informative features. Furthermore, the Pixel-level Feature Enhancement (PFE) module enhances low-level features extracted at shallow network layers by incorporating refined high-level features. This integration ensures the preservation of domain-invariant information, effectively addressing domain variation and mitigating the loss of global features. Two publicly accessible fundus image databases are employed to demonstrate the effectiveness of our MAAL method in mitigating model degradation and improving segmentation performance. The achieved results outperform current state-of-the-art (SOTA) methods in both OD and OC segmentation.

TriLA: Triple-Level Alignment Based Unsupervised Domain Adaptation for Joint Segmentation of Optic Disc and Optic Cup

Cross-domain joint segmentation of optic disc and optic cup on fundus images is essential, yet challenging, for effective glaucoma screening. Although many unsupervised domain adaptation (UDA) methods have been proposed, these methods can hardly achieve complete domain alignment, leading to suboptimal performance. In this paper, we propose a triple-level alignment (TriLA) model to address this issue by aligning the source and target domains at the input level, feature level, and output level simultaneously. At the input level, a learnable Fourier domain adaptation (LFDA) module is developed to learn the cut-off frequency adaptively for frequency-domain translation. At the feature level, we disentangle the style and content features and align them in the corresponding feature spaces using consistency constraints. At the output level, we design a segmentation consistency constraint to emphasize the segmentation consistency across domains. The proposed model is trained on the RIGA+ dataset and widely evaluated on six different UDA scenarios. Our comprehensive results not only demonstrate that the proposed TriLA substantially outperforms other state-of-the-art UDA methods in joint segmentation of optic disc and optic cup, but also suggest the effectiveness of the triple-level alignment strategy.

Highly Accurate and Precise Automated Cup-to-Disc Ratio Quantification for Glaucoma Screening

Objective

An enlarged cup-to-disc ratio (CDR) is a hallmark of glaucomatous optic neuropathy. Manual assessment of the CDR may be less accurate and more time-consuming than automated methods. Here, we sought to develop and validate a deep learning-based algorithm to automatically determine the CDR from fundus images.

Design

Algorithm development for estimating CDR using fundus data from a population-based observational study.

Participants

A total of 181 768 fundus images from the United Kingdom Biobank (UKBB), Drishti_GS, and EyePACS.

Methods

FastAI and PyTorch libraries were used to train a convolutional neural network-based model on fundus images from the UKBB. Models were constructed to determine image gradability (classification analysis) as well as to estimate CDR (regression analysis). The best-performing model was then validated for use in glaucoma screening using a multiethnic dataset from EyePACS and Drishti_GS.

Main Outcome Measures

The area under the receiver operating characteristic curve and coefficient of determination.

Results

Our gradability model vgg19_batch normalization (bn) achieved an accuracy of 97.13% on a validation set of 16 045 images, with 99.26% precision and area under the receiver operating characteristic curve of 96.56%. Using regression analysis, our best-performing model (trained on the vgg19_bn architecture) attained a coefficient of determination of 0.8514 (95% confidence interval [CI]: 0.8459-0.8568), while the mean squared error was 0.0050 (95% CI: 0.0048-0.0051) and mean absolute error was 0.0551 (95% CI: 0.0543-0.0559) on a validation set of 12 183 images for determining CDR. The regression point was converted into classification metrics using a tolerance of 0.2 for 20 classes; the classification metrics achieved an accuracy of 99.20%. The EyePACS dataset (98 172 healthy, 3270 glaucoma) was then used to externally validate the model for glaucoma classification, with an accuracy, sensitivity, and specificity of 82.49%, 72.02%, and 82.83%, respectively.

Conclusions

Our models were precise in determining image gradability and estimating CDR. Although our artificial intelligence-derived CDR estimates achieve high accuracy, the CDR threshold for glaucoma screening will vary depending on other clinical parameters.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Hybrid convolutional neural network optimized with an artificial algae algorithm for glaucoma screening using fundus images

OBJECTIVE

We developed an optimized decision support system for retinal fundus image-based glaucoma screening.

METHODS

We combined computer vision algorithms with a convolutional network for fundus images and applied a faster region-based convolutional neural network (FRCNN) and artificial algae algorithm with support vector machine (AAASVM) classifiers. Optic boundary detection, optic cup, and optic disc segmentations were conducted using TernausNet. Glaucoma screening was performed using the optimized FRCNN. The Softmax layer was replaced with an SVM classifier layer and optimized with an AAA to attain enhanced accuracy.

RESULTS

Using three retinal fundus image datasets (G1020, digital retinal images vessel extraction, and high-resolution fundus), we obtained accuracy of 95.11%, 92.87%, and 93.7%, respectively. Framework accuracy was amplified with an adaptive gradient algorithm optimizer FRCNN (AFRCNN), which achieved average accuracy 94.06%, sensitivity 93.353%, and specificity 94.706%. AAASVM obtained average accuracy of 96.52%, which was 3% ahead of the FRCNN classifier. These classifiers had areas under the curve of 0.9, 0.85, and 0.87, respectively.

CONCLUSION

Based on statistical Friedman evaluation, AAASVM was the best glaucoma screening model. Segmented and classified images can be directed to the health care system to assess patients' progress. This computer-aided decision support system will be useful for optometrists.

Automated Alzheimer's, Mild Cognitive Impairment, and Normal Aging Screening using Polar Transformation of Optic Disc and Central Zone of Fundus Images

Detecting Mild Cognitive Impairment (MCI) is crucial for mitigating the risk of Alzheimer's disease (AD), a leading global cause of death. However, the current gold standard for AD and MCI detection relies on specialized equipment often limited to large testing centers, particularly in low-resource settings like Thailand. Our previous work aimed to create a cost-effective MCI and AD screening method using fundus images but struggled to differentiate between AD and MCI. Henceforth, we developed the proposed methodology, utilizing DenseNet-121 on polar-transformed and zone-selected fundus images, which significantly enhances AD and MCI classification, achieving 83% accuracy, 90% sensitivity, 77% specificity, 87% precision, and an F-1 score of 88%. Moreover, the model's Grad-Cam++ heatmap highlights vasculature differences, particularly in tortuosity and thickness, between AD and MCI fundus images. Combined with our previous work, we created a fully automated pipeline model for MCI, AD, and Normal aging classification, which is inexpensive, fast, and non-invasive with an overall 3-class accuracy of 88%.

Multi-granularity learning of explicit geometric constraint and contrast for label-efficient medical image segmentation and differentiable clinical function assessment

Automated segmentation is a challenging task in medical image analysis that usually requires a large amount of manually labeled data. However, most current supervised learning based algorithms suffer from insufficient manual annotations, posing a significant difficulty for accurate and robust segmentation. In addition, most current semi-supervised methods lack explicit representations of geometric structure and semantic information, restricting segmentation accuracy. In this work, we propose a hybrid framework to learn polygon vertices, region masks, and their boundaries in a weakly/semi-supervised manner that significantly advances geometric and semantic representations. Firstly, we propose multi-granularity learning of explicit geometric structure constraints via polygon vertices (PolyV) and pixel-wise region (PixelR) segmentation masks in a semi-supervised manner. Secondly, we propose eliminating boundary ambiguity by using an explicit contrastive objective to learn a discriminative feature space of boundary contours at the pixel level with limited annotations. Thirdly, we exploit the task-specific clinical domain knowledge to differentiate the clinical function assessment end-to-end. The ground truth of clinical function assessment, on the other hand, can serve as auxiliary weak supervision for PolyV and PixelR learning. We evaluate the proposed framework on two tasks, including optic disc (OD) and cup (OC) segmentation along with vertical cup-to-disc ratio (vCDR) estimation in fundus images; left ventricle (LV) segmentation at end-diastolic and end-systolic frames along with ejection fraction (LVEF) estimation in two-dimensional echocardiography images. Experiments on nine large-scale datasets of the two tasks under different label settings demonstrate our model's superior performance on segmentation and clinical function assessment.

JOINEDTrans: Prior guided multi-task transformer for joint optic disc/cup segmentation and fovea detection

Deep learning-based image segmentation and detection models have largely improved the efficiency of analyzing retinal landmarks such as optic disc (OD), optic cup (OC), and fovea. However, factors including ophthalmic disease-related lesions and low image quality issues may severely complicate automatic OD/OC segmentation and fovea detection. Most existing works treat the identification of each landmark as a single task, and take into account no prior information. To address these issues, we propose a prior guided multi-task transformer framework for joint OD/OC segmentation and fovea detection, named JOINEDTrans. JOINEDTrans effectively combines various spatial features of the fundus images, relieving the structural distortions induced by lesions and other imaging issues. It contains a segmentation branch and a detection branch. To be noted, we employ an encoder with prior-learning in a vessel segmentation task to effectively exploit the positional relationship among vessel, OD/OC, and fovea, successfully incorporating spatial prior into the proposed JOINEDTrans framework. There are a coarse stage and a fine stage in JOINEDTrans. In the coarse stage, OD/OC coarse segmentation and fovea heatmap localization are obtained through a joint segmentation and detection module. In the fine stage, we crop regions of interest for subsequent refinement and use predictions obtained in the coarse stage to provide additional information for better performance and faster convergence. Experimental results demonstrate that JOINEDTrans outperforms existing state-of-the-art methods on the publicly available GAMMA, REFUGE, and PALM fundus image datasets. We make our code available at https://github.com/HuaqingHe/JOINEDTrans.

Research on segmentation model of optic disc and optic cup in fundus

BACKGROUND

Glaucoma is a worldwide eye disease that can cause irreversible vision loss. Early detection of glaucoma is important to reduce vision loss, and retinal fundus image examination is one of the most commonly used solutions for glaucoma diagnosis due to its low cost. Clinically, the cup-disc ratio of fundus images is an important indicator for glaucoma diagnosis. In recent years, there have been an increasing number of algorithms for segmentation and recognition of the optic disc (OD) and optic cup (OC), but these algorithms generally have poor universality, segmentation performance, and segmentation accuracy.

METHODS

By improving the YOLOv8 algorithm for segmentation of OD and OC. Firstly, a set of algorithms was designed to adapt the REFUGE dataset's result images to the input format of the YOLOv8 algorithm. Secondly, in order to improve segmentation performance, the network structure of YOLOv8 was improved, including adding a ROI (Region of Interest) module, modifying the bounding box regression loss function from CIOU to Focal-EIoU. Finally, by training and testing the REFUGE dataset, the improved YOLOv8 algorithm was evaluated.

RESULTS

The experimental results show that the improved YOLOv8 algorithm achieves good segmentation performance on the REFUGE dataset. In the OD and OC segmentation tests, the F1 score is 0.999.

CONCLUSIONS

We improved the YOLOv8 algorithm and applied the improved model to the segmentation task of OD and OC in fundus images. The results show that our improved model is far superior to the mainstream U-Net model in terms of training speed, segmentation performance, and segmentation accuracy.

Robust PCA with Lw,∗ and L2,1 Norms: A Novel Method for Low-Quality Retinal Image Enhancement

Nonmydriatic retinal fundus images often suffer from quality issues and artifacts due to ocular or systemic comorbidities, leading to potential inaccuracies in clinical diagnoses. In recent times, deep learning methods have been widely employed to improve retinal image quality. However, these methods often require large datasets and lack robustness in clinical settings. Conversely, the inherent stability and adaptability of traditional unsupervised learning methods, coupled with their reduced reliance on extensive data, render them more suitable for real-world clinical applications, particularly in the limited data context of high noise levels or a significant presence of artifacts. However, existing unsupervised learning methods encounter challenges such as sensitivity to noise and outliers, reliance on assumptions like cluster shapes, and difficulties with scalability and interpretability, particularly when utilized for retinal image enhancement. To tackle these challenges, we propose a novel robust PCA (RPCA) method with low-rank sparse decomposition that also integrates affine transformations τi, weighted nuclear norm, and the L2,1 norms, aiming to overcome existing method limitations and to achieve image quality improvement unseen by these methods. We employ the weighted nuclear norm (Lw,∗) to assign weights to singular values to each retinal images and utilize the L2,1 norm to eliminate correlated samples and outliers in the retinal images. Moreover, τi is employed to enhance retinal image alignment, making the new method more robust to variations, outliers, noise, and image blurring. The Alternating Direction Method of Multipliers (ADMM) method is used to optimally determine parameters, including τi, by solving an optimization problem. Each parameter is addressed separately, harnessing the benefits of ADMM. Our method introduces a novel parameter update approach and significantly improves retinal image quality, detecting cataracts, and diabetic retinopathy. Simulation results confirm our method's superiority over existing state-of-the-art methods across various datasets.

Entropy and distance-guided super self-ensembling for optic disc and cup segmentation

Segmenting the optic disc (OD) and optic cup (OC) is crucial to accurately detect changes in glaucoma progression in the elderly. Recently, various convolutional neural networks have emerged to deal with OD and OC segmentation. Due to the domain shift problem, achieving high-accuracy segmentation of OD and OC from different domain datasets remains highly challenging. Unsupervised domain adaptation has taken extensive focus as a way to address this problem. In this work, we propose a novel unsupervised domain adaptation method, called entropy and distance-guided super self-ensembling (EDSS), to enhance the segmentation performance of OD and OC. EDSS is comprised of two self-ensembling models, and the Gaussian noise is added to the weights of the whole network. Firstly, we design a super self-ensembling (SSE) framework, which can combine two self-ensembling to learn more discriminative information about images. Secondly, we propose a novel exponential moving average with Gaussian noise (G-EMA) to enhance the robustness of the self-ensembling framework. Thirdly, we propose an effective multi-information fusion strategy (MFS) to guide and improve the domain adaptation process. We evaluate the proposed EDSS on two public fundus image datasets RIGA+ and REFUGE. Large amounts of experimental results demonstrate that the proposed EDSS outperforms state-of-the-art segmentation methods with unsupervised domain adaptation, e.g., the Dicemean score on three test sub-datasets of RIGA+ are 0.8442, 0.8772 and 0.9006, respectively, and the Dicemean score on the REFUGE dataset is 0.9154.

Abdominal multi-organ segmentation in Multi-sequence MRIs based on visual attention guided network and knowledge distillation

PURPOSE

The segmentation of abdominal organs in magnetic resonance imaging (MRI) plays a pivotal role in various therapeutic applications. Nevertheless, the application of deep-learning methods to abdominal organ segmentation encounters numerous challenges, especially in addressing blurred boundaries and regions characterized by low-contrast.

METHODS

In this study, a multi-scale visual attention-guided network (VAG-Net) was proposed for abdominal multi-organ segmentation based on unpaired multi-sequence MRI. A new visual attention-guided (VAG) mechanism was designed to enhance the extraction of contextual information, particularly at the edge of organs. Furthermore, a new loss function inspired by knowledge distillation was introduced to minimize the semantic disparity between different MRI sequences.

RESULTS

The proposed method was evaluated on the CHAOS 2019 Challenge dataset and compared with six state-of-the-art methods. The results demonstrated that our model outperformed these methods, achieving DSC values of 91.83 ± 0.24% and 94.09 ± 0.66% for abdominal multi-organ segmentation in T1-DUAL and T2-SPIR modality, respectively.

CONCLUSION

The experimental results show that our proposed method has superior performance in abdominal multi-organ segmentation, especially in the case of small organs such as the kidneys.

Recurrent attention U-Net for segmentation and quantification of breast arterial calcifications on synthesized 2D mammograms

Breast arterial calcifications (BAC) are a type of calcification commonly observed on mammograms and are generally considered benign and not associated with breast cancer. However, there is accumulating observational evidence of an association between BAC and cardiovascular disease, the leading cause of death in women. We present a deep learning method that could assist radiologists in detecting and quantifying BAC in synthesized 2D mammograms. We present a recurrent attention U-Net model consisting of encoder and decoder modules that include multiple blocks that each use a recurrent mechanism, a recurrent mechanism, and an attention module between them. The model also includes a skip connection between the encoder and the decoder, similar to a U-shaped network. The attention module was used to enhance the capture of long-range dependencies and enable the network to effectively classify BAC from the background, whereas the recurrent blocks ensured better feature representation. The model was evaluated using a dataset containing 2,000 synthesized 2D mammogram images. We obtained 99.8861% overall accuracy, 69.6107% sensitivity, 66.5758% F-1 score, and 59.5498% Jaccard coefficient, respectively. The presented model achieved promising performance compared with related models.

PallorMetrics: Software for Automatically Quantifying Optic Disc Pallor in Fundus Photographs, and Associations With Peripapillary RNFL Thickness

Purpose

We sough to develop an automatic method of quantifying optic disc pallor in fundus photographs and determine associations with peripapillary retinal nerve fiber layer (pRNFL) thickness.

Methods

We used deep learning to segment the optic disc, fovea, and vessels in fundus photographs, and measured pallor. We assessed the relationship between pallor and pRNFL thickness derived from optical coherence tomography scans in 118 participants. Separately, we used images diagnosed by clinical inspection as pale (n = 45) and assessed how measurements compared with healthy controls (n = 46). We also developed automatic rejection thresholds and tested the software for robustness to camera type, image format, and resolution.

Results

We developed software that automatically quantified disc pallor across several zones in fundus photographs. Pallor was associated with pRNFL thickness globally (β = -9.81; standard error [SE] = 3.16; P < 0.05), in the temporal inferior zone (β = -29.78; SE = 8.32; P < 0.01), with the nasal/temporal ratio (β = 0.88; SE = 0.34; P < 0.05), and in the whole disc (β = -8.22; SE = 2.92; P < 0.05). Furthermore, pallor was significantly higher in the patient group. Last, we demonstrate the analysis to be robust to camera type, image format, and resolution.

Conclusions

We developed software that automatically locates and quantifies disc pallor in fundus photographs and found associations between pallor measurements and pRNFL thickness.

Translational Relevance

We think our method will be useful for the identification, monitoring, and progression of diseases characterized by disc pallor and optic atrophy, including glaucoma, compression, and potentially in neurodegenerative disorders.

ICF-PR-Net: a deep phase retrieval neural network for X-ray phase contrast imaging of inertial confinement fusion capsules

X-ray phase contrast imaging (XPCI) has demonstrated capability to characterize inertial confinement fusion (ICF) capsules, and phase retrieval can reconstruct phase information from intensity images. This study introduces ICF-PR-Net, a novel deep learning-based phase retrieval method for ICF-XPCI. We numerically constructed datasets based on ICF capsule shape features, and proposed an object-image loss function to add image formation physics to network training. ICF-PR-Net outperformed traditional methods as it exhibited satisfactory robustness against strong noise and nonuniform background and was well-suited for ICF-XPCI's constrained experimental conditions and single exposure limit. Numerical and experimental results showed that ICF-PR-Net accurately retrieved the phase and absorption while maintaining retrieval quality in different situations. Overall, the ICF-PR-Net enables the diagnosis of the inner interface and electron density of capsules to address ignition-preventing problems, such as hydrodynamic instability growth.

Generalizability of Deep Neural Networks for Vertical Cup-to-Disc Ratio Estimation in Ultra-Widefield and Smartphone-Based Fundus Images

Purpose

To develop and validate a deep learning system (DLS) for estimation of vertical cup-to-disc ratio (vCDR) in ultra-widefield (UWF) and smartphone-based fundus images.

Methods

A DLS consisting of two sequential convolutional neural networks (CNNs) to delineate optic disc (OD) and optic cup (OC) boundaries was developed using 800 standard fundus images from the public REFUGE data set. The CNNs were tested on 400 test images from the REFUGE data set and 296 UWF and 300 smartphone-based images from a teleophthalmology clinic. vCDRs derived from the delineated OD/OC boundaries were compared with optometrists' annotations using mean absolute error (MAE). Subgroup analysis was conducted to study the impact of peripapillary atrophy (PPA), and correlation study was performed to investigate potential correlations between sectoral CDR (sCDR) and retinal nerve fiber layer (RNFL) thickness.

Results

The system achieved MAEs of 0.040 (95% CI, 0.037-0.043) in the REFUGE test images, 0.068 (95% CI, 0.061-0.075) in the UWF images, and 0.084 (95% CI, 0.075-0.092) in the smartphone-based images. There was no statistical significance in differences between PPA and non-PPA images. Weak correlation (r = -0.4046, P < 0.05) between sCDR and RNFL thickness was found only in the superior sector.

Conclusions

We developed a deep learning system that estimates vCDR from standard, UWF, and smartphone-based images. We also described anatomic peripapillary adversarial lesion and its potential impact on OD/OC delineation.

Translational Relevance

Artificial intelligence can estimate vCDR from different types of fundus images and may be used as a general and interpretable screening tool to improve community reach for diagnosis and management of glaucoma.

Optic disc and cup segmentation for glaucoma detection using Attention U-Net incorporating residual mechanism

Glaucoma is a common eye disease that can cause blindness. Accurate detection of the optic disc and cup disc is crucial for glaucoma diagnosis. Algorithm models based on artificial intelligence can assist doctors in improving detection performance. In this article, U-Net is used as the backbone network, and the attention and residual modules are integrated to construct an end-to-end convolutional neural network model for optic disc and cup disc segmentation. The U-Net backbone is used to infer the basic position information of optic disc and cup disc, the attention module enhances the model's ability to represent and extract features of optic disc and cup disc, and the residual module alleviates gradient disappearance or explosion that may occur during feature representation of the neural network. The proposed model is trained and tested on the DRISHTI-GS1 dataset. Results show that compared with the original U-Net method, our model can more effectively separate optic disc and cup disc in terms of overlap error, sensitivity, and specificity.

Self-supervised pre-training for joint optic disc and cup segmentation via attention-aware network

Image segmentation is a fundamental task in deep learning, which is able to analyse the essence of the images for further development. However, for the supervised learning segmentation method, collecting pixel-level labels is very time-consuming and labour-intensive. In the medical image processing area for optic disc and cup segmentation, we consider there are two challenging problems that remain unsolved. One is how to design an efficient network to capture the global field of the medical image and execute fast in real applications. The other is how to train the deep segmentation network using a few training data due to some medical privacy issues. In this paper, to conquer such issues, we first design a novel attention-aware segmentation model equipped with the multi-scale attention module in the pyramid structure-like encoder-decoder network, which can efficiently learn the global semantics and the long-range dependencies of the input images. Furthermore, we also inject the prior knowledge that the optic cup lies inside the optic disc by a novel loss function. Then, we propose a self-supervised contrastive learning method for optic disc and cup segmentation. The unsupervised feature representation is learned by matching an encoded query to a dictionary of encoded keys using a contrastive technique. Finetuning the pre-trained model using the proposed loss function can help achieve good performance for the task. To validate the effectiveness of the proposed method, extensive systemic evaluations on different public challenging optic disc and cup benchmarks, including DRISHTI-GS and REFUGE datasets demonstrate the superiority of the proposed method, which can achieve new state-of-the-art performance approaching 0.9801 and 0.9087 F1 score respectively while gaining 0.9657 D C disc and 0.8976 D C cup . The code will be made publicly available.

A novel lightweight deep learning approach for simultaneous optic cup and optic disc segmentation in glaucoma detection

Glaucoma is a chronic neurodegenerative disease that can result in irreversible vision loss if not treated in its early stages. The cup-to-disc ratio is a key criterion for glaucoma screening and diagnosis, and it is determined by dividing the area of the optic cup (OC) by that of the optic disc (OD) in fundus images. Consequently, the automatic and accurate segmentation of the OC and OD is a pivotal step in glaucoma detection. In recent years, numerous methods have resulted in great success on this task. However, most existing methods either have unsatisfactory segmentation accuracy or high time costs. In this paper, we propose a lightweight deep-learning architecture for the simultaneous segmentation of the OC and OD, where we have adopted fuzzy learning and a multi-layer perceptron to simplify the learning complexity and improve segmentation accuracy. Experimental results demonstrate the superiority of our proposed method as compared to most state-of-the-art approaches in terms of both training time and segmentation accuracy.

EPolar-UNet: An edge-attending polar UNet for automatic medical image segmentation with small datasets

BACKGROUND

Medical image segmentation is one of the most key steps in computer-aided clinical diagnosis, geometric characterization, measurement, image registration, and so forth. Convolutional neural networks especially UNet and its variants have been successfully used in many medical image segmentation tasks. However, the results are limited by the deficiency in extracting high resolution edge information because of the design of the skip connections in UNet and the need for large available datasets.

PURPOSE

In this paper, we proposed an edge-attending polar UNet (EPolar-UNet), which was trained on the polar coordinate system instead of classic Cartesian coordinate system with an edge-attending construction in skip connection path.

METHODS

EPolar-UNet extracted the location information from an eight-stacked hourglass network as the pole for polar transformation and extracted the boundary cues from an edge-attending UNet, which consisted of a deconvolution layer and a subtraction operation.

RESULTS

We evaluated the performance of EPolar-UNet across three imaging modalities for different segmentation tasks: CVC-ClinicDB dataset for polyp, ISIC-2018 dataset for skin lesion, and our private ultrasound dataset for liver tumor segmentation. Our proposed model outperformed state-of-the-art models on all three datasets and needed only 30%-60% of training data compared with the benchmark UNet model to achieve similar performances for medical image segmentation tasks.

CONCLUSIONS

We proposed an end-to-end EPolar-UNet for automatic medical image segmentation and showed good performance on small datasets, which was critical in the field of medical image segmentation.

TdDS-UNet: top-down deeply supervised U-Net for the delineation of 3D colorectal cancer

Automatically delineating colorectal cancers with fuzzy boundaries from 3D images is a challenging task, but the problem of fuzzy boundary delineation in existing deep learning-based methods have not been investigated in depth. Here, an encoder-decoder-based U-shaped network (U-Net) based on top-down deep supervision (TdDS) was designed to accurately and automatically delineate the fuzzy boundaries of colorectal cancer. TdDS refines the semantic targets of the upper and lower stages by mapping ground truths that are more consistent with the stage properties than upsampling deep supervision. This stage-specific approach can guide the model to learn a coarse-to-fine delineation process and improve the delineation accuracy of fuzzy boundaries by gradually shrinking the boundaries. Experimental results showed that TdDS is more customizable and plays a role similar to the attentional mechanism, and it can further improve the capability of the model to delineate colorectal cancer contours. A total of 103, 12, and 29 3D pelvic magnetic resonance imaging volumes were used for training, validation, and testing, respectively. The comparative results indicate that the proposed method exhibits the best comprehensive performance, with a dice similarity coefficient (DSC) of 0.805 ± 0.053 and a hausdorff distance (HD) of 9.28 ± 5.14 voxels. In the delineation performance analysis section also showed that 44.49% of the delineation results are satisfactory and do not require revisions. This study can provide new technical support for the delineation of 3D colorectal cancer. Our method is open source, and the code is available athttps://github.com/odindis/TdDS/tree/main.

Deep-learning segmentation method for optical coherence tomography angiography in ophthalmology

PURPOSE

The optic disc and the macular are two major anatomical structures in the human eye. Optic discs are associated with the optic nerve. Macular mainly involves degeneration and impaired function of the macular region. Reliable optic disc and macular segmentation are necessary for the automated screening of retinal diseases.

METHODS

A swept-source OCTA system was designed to capture OCTA images of human eyes. To address these segmentation tasks, first, we constructed a new Optic Disc and Macula in fundus Image with optical coherence tomography angiography (OCTA) dataset (ODMI). Second, we proposed a Coarse and Fine Attention-Based Network (CFANet).

RESULTS

The five metrics of our methods on ODMI are 98.91 % , 98.47 % , 89.77 % , 98.49 % , and 89.77 % , respectively.

CONCLUSIONS

Experimental results show that our CFANet has achieved good performance on segmentation for the optic disc and macula in OCTA.

Lightweight and multi-lesion segmentation model for diabetic retinopathy based on the fusion of mixed attention and ghost feature mapping

Diabetic retinopathy is the main cause of blindness, and lesion segmentation is an important basic work for the diagnosis of this disease. The main lesions include soft and hard exudates, microaneurysms, and hemorrhages. However, the segmentation of these four types of lesions is difficult because of their variability in size and contrast, and high intertype similarity. Currently, many network models have problems, such as a large number of parameters and complex calculations, and most segmentation models for diabetic retinopathy focus only on one type of lesion. In this study, a lightweight algorithm based on BiSeNet V2 was proposed for the segmentation of multiple lesions in diabetic retinopathy fundus. First, a hybrid attention module was embedded in the semantic branch of BiSeNet V2 for 8- and 16-fold downsampling, which helped reassign deep feature-map weights and enhanced the ability to extract local key features. Second, a ghost feature-mapping unit was used to optimize the traditional convolution layers and further reduce the computational cost. Third, a new loss function based on the dynamic threshold loss function was applied to supervise the training by adjusting the training weights of the high-loss difficult samples, which enhanced the model's attention to small goals. In experiments on the IDRiD dataset, we conducted an ablation study to verify the effectiveness of each component and compared the proposed model, BiSeNet V2-Pro, with several state-of-the-art models. In comparison with the baseline BiSeNet V2, the segmentation performance of BiSeNet V2-Pro improved by 12.17 %, 11.44 %, and 8.49 % in terms of Sensitivity (SEN), Intersection over Union (IoU), and Dice coefficient (DICE), respectively. Specifically, IoU of MA reaches 0.5716. Compared with other methods, the segmentation speed was significantly improved while ensuring segmentation accuracy, and the number of model parameters was lower. These results demonstrate the superiority of BiSeNet V2-Pro in the multi-lesion segmentation of diabetic retinopathy.

W-Net: A boundary-aware cascade network for robust and accurate optic disc segmentation

Accurate optic disc (OD) segmentation has a great significance for computer-aided diagnosis of different types of eye diseases. Due to differences in image acquisition equipment and acquisition methods, the resolution, size, contrast, and clarity of images from different datasets show significant differences, resulting in poor generalization performance of deep learning networks. To solve this problem, this study proposes a multi-level segmentation network. The network includes data quality enhancement module (DQEM), coarse segmentation module (CSM), localization module (OLM), and fine segmentation stage module (FSM). In FSM, W-Net is proposed for the first time, and boundary loss is introduced in the loss function, which effectively improves the performance of OD segmentation. We generalized the model in the REFUGE test dataset, GAMMA dataset, Drishti-GS1 dataset, and IDRiD dataset, respectively. The results show that our method has the best OD segmentation performance in different datasets compared with state-of-the-art networks.

Multi-dimensional dense attention network for pixel-wise segmentation of optic disc in colour fundus images

BACKGROUND

Segmentation of retinal fragments like blood vessels, Optic Disc (OD), and Optic Cup (OC) enables the early detection of different retinal pathologies like Diabetic Retinopathy (DR), Glaucoma, etc.

OBJECTIVE

Accurate segmentation of OD remains challenging due to blurred boundaries, vessel occlusion, and other distractions and limitations. These days, deep learning is rapidly progressing in the segmentation of image pixels, and a number of network models have been proposed for end-to-end image segmentation. However, there are still certain limitations, such as limited ability to represent context, inadequate feature processing, limited receptive field, etc., which lead to the loss of local details and blurred boundaries.

METHODS

A multi-dimensional dense attention network, or MDDA-Net, is proposed for pixel-wise segmentation of OD in retinal images in order to address the aforementioned issues and produce more thorough and accurate segmentation results. In order to acquire powerful contexts when faced with limited context representation capabilities, a dense attention block is recommended. A triple-attention (TA) block is introduced in order to better extract the relationship between pixels and obtain more comprehensive information, with the goal of addressing the insufficient feature processing. In the meantime, a multi-scale context fusion (MCF) is suggested for acquiring the multi-scale contexts through context improvement.

RESULTS

Specifically, we provide a thorough assessment of the suggested approach on three difficult datasets. In the MESSIDOR and ORIGA data sets, the suggested MDDA-NET approach obtains accuracy levels of 99.28% and 98.95%, respectively.

CONCLUSION

The experimental results show that the MDDA-Net can obtain better performance than state-of-the-art deep learning models under the same environmental conditions.

Optic disc detection based on fully convolutional network and weighted matrix recovery model

Eye diseases often affect human health. Accurate detection of the optic disc contour is one of the important steps in diagnosing and treating eye diseases. However, the structure of fundus images is complex, and the optic disc region is often disturbed by blood vessels. Considering that the optic disc is usually a saliency region in fundus images, we propose a weakly-supervised optic disc detection method based on the fully convolution neural network (FCN) combined with the weighted low-rank matrix recovery model (WLRR). Firstly, we extract the low-level features of the fundus image and cluster the pixels using the Simple Linear Iterative Clustering (SLIC) algorithm to generate the feature matrix. Secondly, the top-down semantic prior information provided by FCN and bottom-up background prior information of the optic disc region are used to jointly construct the prior information weighting matrix, which more accurately guides the decomposition of the feature matrix into a sparse matrix representing the optic disc and a low-rank matrix representing the background. Experimental results on the DRISHTI-GS dataset and IDRiD dataset show that our method can segment the optic disc region accurately, and its performance is better than existing weakly-supervised optic disc segmentation methods. Graphical abstract of optic disc segmentation.

Facial wrinkle segmentation using weighted deep supervision and semi-automatic labeling

Facial wrinkles are important indicators of human aging. Recently, a method using deep learning and a semi-automatic labeling was proposed to segment facial wrinkles, which showed much better performance than conventional image-processing-based methods. However, the difficulty of wrinkle segmentation remains challenging due to the thinness of wrinkles and their small proportion in the entire image. Therefore, performance improvement in wrinkle segmentation is still necessary. To address this issue, we propose a novel loss function that takes into account the thickness of wrinkles based on the semi-automatic labeling approach. First, considering the different spatial dimensions of the decoder in the U-Net architecture, we generated weighted wrinkle maps from ground truth. These weighted wrinkle maps were used to calculate the training losses more accurately than the existing deep supervision approach. This new loss computation approach is defined as weighted deep supervision in our study. The proposed method was evaluated using an image dataset obtained from a professional skin analysis device and labeled using semi-automatic labeling. In our experiment, the proposed weighted deep supervision showed higher Jaccard Similarity Index (JSI) performance for wrinkle segmentation compared to conventional deep supervision and traditional image processing methods. Additionally, we conducted experiments on the labeling using a semi-automatic labeling approach, which had not been explored in previous research, and compared it with human labeling. The semi-automatic labeling technology showed more consistent wrinkle labels than human-made labels. Furthermore, to assess the scalability of the proposed method to other domains, we applied it to retinal vessel segmentation. The results demonstrated superior performance of the proposed method compared to existing retinal vessel segmentation approaches. In conclusion, the proposed method offers high performance and can be easily applied to various biomedical domains and U-Net-based architectures. Therefore, the proposed approach will be beneficial for various biomedical imaging approaches. To facilitate this, we have made the source code of the proposed method publicly available at: https://github.com/resemin/WeightedDeepSupervision.