Comparative Analysis of Vision Transformers and Conventional Convolutional Neural Networks in Detecting Referable Diabetic Retinopathy

Vision transformer-based stratification of pre/diabetic and pre/hypertensive patients from retinal photographs for 3PM applications

Objective

Diabetes and hypertension pose significant health risks, especially when poorly managed. Retinal evaluation though fundus photography can provide non-invasive assessment of these diseases, yet prior studies focused on disease presence, overlooking control statuses. This study evaluated vision transformer (ViT)-based models for assessing the presence and control statuses of diabetes and hypertension from retinal images.

Methods

ViT-based models with ResNet-50 for patch projection were trained on images from the UK Biobank (n = 113,713) and Singapore Epidemiology of Eye Diseases study (n = 17,783), and externally validated on the Singapore Prospective Study Programme (n = 7,793) and the Beijing Eye Study (n = 6064). Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC) for multiple tasks: detecting disease, identifying poorly controlled and well-controlled cases, distinguishing between poorly and well-controlled cases, and detecting pre-diabetes or pre-hypertension.

Results

The models demonstrated strong performance in detecting disease presence, with AUROC values of 0.820 for diabetes and 0.781 for hypertension in internal testing. External validation showed AUROCs ranging from 0.635 to 0.755 for diabetes, and 0.727 to 0.832 for hypertension. For identifying poorly controlled cases, the performance remained high with AUROCs of 0.871 (internal) and 0.655-0.851 (external) for diabetes, and 0.853 (internal) and 0.792-0.915 (external) for hypertension. Detection of well-controlled cases also yielded promising results for diabetes (0.802 [internal]; 0.675-0.838 [external]), and hypertension (0.740 [internal] and 0.675-0.807 [external]). In distinguishing between poorly and well-controlled disease, AUROCs were more modest with 0.630 (internal) and 0.512-0.547 (external) for diabetes, and 0.651 (internal) and 0.639-0.683 (external) for hypertension. For pre-disease detection, the models achieved AUROCs of 0.746 (internal) and 0.523-0.590 (external) for pre-diabetes, and 0.669 (internal) and 0.645-0.679 (external) for pre-hypertension.

Conclusion

ViT-based models show promise in classifying the presence and control statuses of diabetes and hypertension from retinal images. These findings support the potential of retinal imaging as a tool in primary care for opportunistic detection of diabetes and hypertension, risk stratification, and individualised treatment planning. Further validation in diverse clinical settings is warranted to confirm practical utility.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13167-025-00412-9.

Vision transformer-based multimodal fusion network for classification of tumor malignancy on breast ultrasound: A retrospective multicenter study

BACKGROUND

In the context of routine breast cancer diagnosis, the precise discrimination between benign and malignant breast masses holds utmost significance. Notably, few prior investigations have concurrently explored the integration of imaging histology features, deep learning characteristics, and clinical parameters. The primary objective of this retrospective study was to pioneer a multimodal feature fusion model tailored for the prediction of breast tumor malignancy, harnessing the potential of ultrasound images.

METHOD

We compiled a dataset that included clinical features from 1065 patients and 3315 image datasets. Specifically, we selected data from 603 patients for training our multimodal model. The comprehensive experimental workflow involves identifying the optimal unimodal model, extracting unimodal features, fusing multimodal features, gaining insights from these fused features, and ultimately generating prediction results using a classifier.

RESULTS

Our multimodal feature fusion model demonstrates outstanding performance, achieving an AUC of 0.994 (95 % CI: 0.988-0.999) and an F1 score of 0.971 on the primary multicenter dataset. In the evaluation on two independent testing cohorts (TCs), it maintains strong performance, with AUCs of 0.942 (95 % CI: 0.854-0.994) for TC1 and 0.945 (95 % CI: 0.857-1.000) for TC2, accompanied by corresponding F1 scores of 0.872 and 0.857, respectively. Notably, the decision curve analysis reveals that our model achieves higher accuracy within the threshold probability range of approximately [0.210, 0.890] (TC1) and [0.000, 0.850] (TC2) compared to alternative methods. This capability enhances its utility in clinical decision-making, providing substantial benefits.

CONCLUSION

The multimodal model proposed in this paper can comprehensively evaluate patients' multifaceted clinical information, achieve the prediction of benign and malignant breast ultrasound tumors, and obtain high performance indexes.

Updates on medical and surgical managements of diabetic retinopathy and maculopathy

Diabetic retinopathy (DR) and diabetic macular edema (DME) are leading causes of vision loss globally. This is a comprehensive review focused on both medical and surgical management strategies for DR and DME. This review highlights the epidemiology of DR and DME, with a particular emphasis on the Asia-Pacific region, urban-rural disparities, ethnic variations, and grading methodologies. We examine various risk factors for DR, including glycemic control, hypertension, hyperlipidemia, obesity, chronic kidney disease, sex, myopia, pregnancy, and cataract surgery. Furthermore, we explore potential biomarkers in serum, proteomics, metabolomics, vitreous, microRNA, and genetics that may aid in the detection and management of DR. In addition to medical management, we review the evidence supporting systemic and ocular treatments for DR/DME, including anti-vascular endothelial growth factor (anti-VEGF) agents, anti-inflammatory agents, biosimilars, and integrin inhibitors. Despite advancements in treatment options such as pan-retinal photocoagulation and anti-VEGF agents, a subset of cases still progresses, necessitating vitrectomy. Challenging diabetic vitrectomies pose difficulties due to complex fibrovascular proliferations, incomplete posterior vitreous detachment, and fragile, ischemic retinas, making membrane dissection risky and potentially damaging to the retina. In this review, we address the question of challenging diabetic vitrectomies, providing insights and strategies to minimize complications. Additionally, we briefly explore newer modalities such as 3-dimensional vitrectomy and intra-operative optical coherence tomography as potential tools in diabetic vitrectomy. In conclusion, this review provides a comprehensive overview of both medical and surgical management options for DR and DME. It underscores the importance of a multidisciplinary approach, tailored to the needs of each patient, to optimize visual outcomes and improve the quality of life for those affected by these sight-threatening conditions.