Deep semi-supervised multiple instance learning with self-correction for DME classification from OCT images

Benefit from public unlabeled data: A Frangi filter-based pretraining network for 3D cerebrovascular segmentation

Precise cerebrovascular segmentation in Time-of-Flight Magnetic Resonance Angiography (TOF-MRA) data is crucial for computer-aided clinical diagnosis. The sparse distribution of cerebrovascular structures within TOF-MRA images often results in high costs for manual data labeling. Leveraging unlabeled TOF-MRA data can significantly enhance model performance. In this study, we have constructed the largest preprocessed unlabeled TOF-MRA dataset to date, comprising 1510 subjects. Additionally, we provide manually annotated segmentation masks for 113 subjects based on existing external image datasets to facilitate evaluation. We propose a simple yet effective pretraining strategy utilizing the Frangi filter, known for its capability to enhance vessel-like structures, to optimize the use of the unlabeled data for 3D cerebrovascular segmentation. This involves a Frangi filter-based preprocessing workflow tailored for large-scale unlabeled datasets and a multi-task pretraining strategy to efficiently utilize the preprocessed data. This approach ensures maximal extraction of useful knowledge from the unlabeled data. The efficacy of the pretrained model is assessed across four cerebrovascular segmentation datasets, where it demonstrates superior performance, improving the clDice metric by approximately 2%-3% compared to the latest semi- and self-supervised methods. Additionally, ablation studies validate the generalizability and effectiveness of our pretraining method across various backbone structures. The code and data have been open source at: https://github.com/shigen-StoneRoot/FFPN.

Anomaly Detection in Retinal OCT Images With Deep Learning-Based Knowledge Distillation

Purpose

The purpose of this study was to develop a robust and general purpose artificial intelligence (AI) system that allows the identification of retinal optical coherence tomography (OCT) volumes with pathomorphological manifestations not present in normal eyes in screening programs and large retrospective studies.

Methods

An unsupervised anomaly detection deep learning approach for the screening of retinal OCTs with any pathomorphological manifestations via Teacher-Student knowledge distillation is developed. The system is trained with only normal cases without any additional manual labeling. At test time, it scores how anomalous a sample is and produces localized anomaly maps with regions of interest in a B-scan. Fovea-centered OCT scans acquired with Spectralis (Heidelberg Engineering) were considered. A total of 3358 patients were used for development and testing. The detection performance was evaluated in a large data cohort with different pathologies including diabetic macular edema (DME) and the multiple stages of age-related macular degeneration (AMD) and on external public datasets with various disease biomarkers.

Results

The volume-wise anomaly detection receiver operating characteristic (ROC) area under the curve (AUC) was 0.94 ± 0.05 in the test set. Pathological B-scan detection on external datasets varied between 0.81 and 0.87 AUC. Qualitatively, the derived anomaly maps pointed toward diagnostically relevant regions. The behavior of the system across the datasets was similar and consistent.

Conclusions

Anomaly detection constitutes a valid complement to supervised systems aimed at improving the success of vision preservation and eye care, and is an important step toward more efficient and generalizable screening tools.

Translational Relevance

Deep learning approaches can enable an automated and objective screening of a wide range of pathological retinal conditions that deviate from normal appearance.

Diversity matters: Cross-head mutual mean-teaching for semi-supervised medical image segmentation

Semi-supervised medical image segmentation (SSMIS) has witnessed substantial advancements by leveraging limited labeled data and abundant unlabeled data. Nevertheless, existing state-of-the-art (SOTA) methods encounter challenges in accurately predicting labels for the unlabeled data, giving rise to disruptive noise during training and susceptibility to erroneous information overfitting. Moreover, applying perturbations to inaccurate predictions further impedes consistent learning. To address these concerns, we propose a novel cross-head mutual mean-teaching network (CMMT-Net) incorporated weak-strong data augmentations, thereby benefiting both co-training and consistency learning. More concretely, our CMMT-Net extends the cross-head co-training paradigm by introducing two auxiliary mean teacher models, which yield more accurate predictions and provide supplementary supervision. The predictions derived from weakly augmented samples generated by one mean teacher are leveraged to guide the training of another student with strongly augmented samples. Furthermore, two distinct yet synergistic data perturbations at the pixel and region levels are introduced. We propose mutual virtual adversarial training (MVAT) to smooth the decision boundary and enhance feature representations, and a cross-set CutMix strategy to generate more diverse training samples for capturing inherent structural data information. Notably, CMMT-Net simultaneously implements data, feature, and network perturbations, amplifying model diversity and generalization performance. Experimental results on three publicly available datasets indicate that our approach yields remarkable improvements over previous SOTA methods across various semi-supervised scenarios. The code is available at https://github.com/Leesoon1984/CMMT-Net.

Identifying retinopathy in optical coherence tomography images with less labeled data via contrastive graph regularization

Retinopathy detection using optical coherence tomography (OCT) images has greatly advanced with computer vision but traditionally requires extensive annotated data, which is time-consuming and expensive. To address this issue, we propose a novel contrastive graph regularization method for detecting retinopathies with less labeled OCT images. This method combines class prediction probabilities and embedded image representations for training, where the two representations interact and co-evolve within the same training framework. Specifically, we leverage memory smoothing constraints to improve pseudo-labels, which are aggregated by nearby samples in the embedding space, effectively reducing overfitting to incorrect pseudo-labels. Our method, using only 80 labeled OCT images, outperforms existing methods on two widely used OCT datasets, with classification accuracy exceeding 0.96 and an Area Under the Curve (AUC) value of 0.998. Additionally, compared to human experts, our method achieves expert-level performance with only 80 labeled images and surpasses most experts with just 160 labeled images.

Global research trends and future directions in diabetic macular edema research: A bibliometric and visualized analysis

BACKGROUND

Diabetic Macular Edema (DME) significantly impairs vision in diabetics, with varied patient responses to current treatments like anti-vascular endothelial growth factor (VEGF) therapy underscoring the necessity for continued research into more effective strategies. This study aims to evaluate global research trends and identify emerging frontiers in DME to guide future research and clinical management.

METHODS

A qualitative and quantitative analysis of publications related to diabetic macular edema retrieved from the Web of Science Core Collection (WoSCC) between its inception and September 4, 2023, was conducted. Microsoft Excel, CiteSpace, VOSviewer, Bibliometrix Package, and Tableau were used for the bibliometric analysis and visualization. This encompasses an examination of the overall distribution of annual output, major countries, regions, institutions, authors, core journals, co-cited references, and keyword analyses.

RESULTS

Overall, 5624 publications were analyzed, indicating an increasing trend in DME research. The United States was identified as the leading country in DME research, with the highest h-index of 135 and 91,841 citations. Francesco Bandello emerged as the most prolific author with 97 publications. Neil M. Bressler has the highest h-index and highest total citation count of 46 and 9692, respectively. The journals "Retina - the Journal of Retinal and Vitreous Diseases" and "Ophthalmology" were highlighted as the most prominent in this field. "Retina" leads with 354 publications, a citation count of 11,872, and an h-index of 59. Meanwhile, "Ophthalmology" stands out with the highest overall citation count of 31,558 and the highest h-index of 90. The primary research focal points in diabetic macular edema included "prevalence and risk factors," "pathological mechanisms," "imaging modalities," "treatment strategies," and "clinical trials." Emerging research areas encompassed "deep learning and artificial intelligence," "novel treatment modalities," and "biomarkers."

CONCLUSION

Our bibliometric analysis delineates the leading role of the United States in DME research. We identified current research hotspots, including epidemiological studies, pathophysiological mechanisms, imaging advancements, and treatment innovations. Emerging trends, such as the integration of artificial intelligence and novel therapeutic approaches, highlight future directions. These insights underscore the importance of collaborative and interdisciplinary approaches in advancing DME research and clinical management.

CLANet: A comprehensive framework for cross-batch cell line identification using brightfield images

Cell line authentication plays a crucial role in the biomedical field, ensuring researchers work with accurately identified cells. Supervised deep learning has made remarkable strides in cell line identification by studying cell morphological features through cell imaging. However, biological batch (bio-batch) effects, a significant issue stemming from the different times at which data is generated, lead to substantial shifts in the underlying data distribution, thus complicating reliable differentiation between cell lines from distinct batch cultures. To address this challenge, we introduce CLANet, a pioneering framework for cross-batch cell line identification using brightfield images, specifically designed to tackle three distinct bio-batch effects. We propose a cell cluster-level selection method to efficiently capture cell density variations, and a self-supervised learning strategy to manage image quality variations, thus producing reliable patch representations. Additionally, we adopt multiple instance learning(MIL) for effective aggregation of instance-level features for cell line identification. Our innovative time-series segment sampling module further enhances MIL's feature-learning capabilities, mitigating biases from varying incubation times across batches. We validate CLANet using data from 32 cell lines across 93 experimental bio-batches from the AstraZeneca Global Cell Bank. Our results show that CLANet outperforms related approaches (e.g. domain adaptation, MIL), demonstrating its effectiveness in addressing bio-batch effects in cell line identification.

Enhancing Readability and Detection of Age-Related Macular Degeneration Using Optical Coherence Tomography Imaging: An AI Approach

Artificial intelligence has been used effectively in medical diagnosis. The objective of this project is to examine the application of a collective AI model using weighted fusion of predicted probabilities from different AI architectures to diagnose various retinal conditions based on optical coherence tomography (OCT). A publicly available Noor dataset, comprising 16,822, images from 554 retinal OCT scans of 441 patients, was used to predict a diverse spectrum of age-related macular degeneration (AMD) stages: normal, drusen, or choroidal neovascularization. These predictions were compared with predictions from ResNet, EfficientNet, and Attention models, respectively, using precision, recall, F1 score, and confusion matric and receiver operating characteristics curves. Our collective model demonstrated superior accuracy in classifying AMD compared to individual ResNet, EfficientNet, and Attention models, showcasing the effectiveness of using trainable weights in the ensemble fusion process, where these weights dynamically adapt during training rather than being fixed values. Specifically, our ensemble model achieved an accuracy of 91.88%, precision of 92.54%, recall of 92.01%, and F1 score of 92.03%, outperforming individual models. Our model also highlights the refinement process undertaken through a thorough examination of initially misclassified cases, leading to significant improvements in the model's accuracy rate to 97%. This study also underscores the potential of AI as a valuable tool in ophthalmology. The proposed ensemble model, combining different mechanisms highlights the benefits of model fusion for complex medical image analysis.

Integrating image and gene-data with a semi-supervised attention model for prediction of KRAS gene mutation status in non-small cell lung cancer

KRAS is a pathogenic gene frequently implicated in non-small cell lung cancer (NSCLC). However, biopsy as a diagnostic method has practical limitations. Therefore, it is important to accurately determine the mutation status of the KRAS gene non-invasively by combining NSCLC CT images and genetic data for early diagnosis and subsequent targeted therapy of patients. This paper proposes a Semi-supervised Multimodal Multiscale Attention Model (S2MMAM). S2MMAM comprises a Supervised Multilevel Fusion Segmentation Network (SMF-SN) and a Semi-supervised Multimodal Fusion Classification Network (S2MF-CN). S2MMAM facilitates the execution of the classification task by transferring the useful information captured in SMF-SN to the S2MF-CN to improve the model prediction accuracy. In SMF-SN, we propose a Triple Attention-guided Feature Aggregation module for obtaining segmentation features that incorporate high-level semantic abstract features and low-level semantic detail features. Segmentation features provide pre-guidance and key information expansion for S2MF-CN. S2MF-CN shares the encoder and decoder parameters of SMF-SN, which enables S2MF-CN to obtain rich classification features. S2MF-CN uses the proposed Intra and Inter Mutual Guidance Attention Fusion (I2MGAF) module to first guide segmentation and classification feature fusion to extract hidden multi-scale contextual information. I2MGAF then guides the multidimensional fusion of genetic data and CT image data to compensate for the lack of information in single modality data. S2MMAM achieved 83.27% AUC and 81.67% accuracy in predicting KRAS gene mutation status in NSCLC. This method uses medical image CT and genetic data to effectively improve the accuracy of predicting KRAS gene mutation status in NSCLC.

Performance of Artificial Intelligence in Detecting Diabetic Macular Edema From Fundus Photography and Optical Coherence Tomography Images: A Systematic Review and Meta-analysis

BACKGROUND

Diabetic macular edema (DME) is the leading cause of vision loss in people with diabetes. Application of artificial intelligence (AI) in interpreting fundus photography (FP) and optical coherence tomography (OCT) images allows prompt detection and intervention.

PURPOSE

To evaluate the performance of AI in detecting DME from FP or OCT images and identify potential factors affecting model performances.

DATA SOURCES

We searched seven electronic libraries up to 12 February 2023.

STUDY SELECTION

We included studies using AI to detect DME from FP or OCT images.

DATA EXTRACTION

We extracted study characteristics and performance parameters.

DATA SYNTHESIS

Fifty-three studies were included in the meta-analysis. FP-based algorithms of 25 studies yielded pooled area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity of 0.964, 92.6%, and 91.1%, respectively. OCT-based algorithms of 28 studies yielded pooled AUROC, sensitivity, and specificity of 0.985, 95.9%, and 97.9%, respectively. Potential factors improving model performance included deep learning techniques, larger size, and more diversity in training data sets. Models demonstrated better performance when validated internally than externally, and those trained with multiple data sets showed better results upon external validation.

LIMITATIONS

Analyses were limited by unstandardized algorithm outcomes and insufficient data in patient demographics, OCT volumetric scans, and external validation.

CONCLUSIONS

This meta-analysis demonstrates satisfactory performance of AI in detecting DME from FP or OCT images. External validation is warranted for future studies to evaluate model generalizability. Further investigations may estimate optimal sample size, effect of class balance, patient demographics, and additional benefits of OCT volumetric scans.