Vision Transformer Architecture for Simultaneous Detection and Severity Grading of Diabetic Retinopathy and Diabetic Macular Edema
Article Sidebar
Main Article Content
Abstract
Diabetic retinopathy (DR) and diabetic macular edema (DME) are two major causes of preventable blindness. Despite their great success in DR and DME detection or grading individually, few methods have explored automated simultaneous grading of DR and DME from fundus photographs in clinic settings. In this work, we introduce DualHead-ViT, a multi-task framework with a pretrained ViT-B/16 backbone and a novel cross-task attention fusion module for jointly grading the severity of DR and DME. We project two separate CLS token vectors for DR and DME, respectively, and design a cross-task attention module to interact features of DR and DME bidirectionally. We validated our model using the Indian Diabetic Retinopathy Image Dataset (IDRID), employing stratified 5-fold cross- validation alongside CLAHE-preprocessed images and a class-weighted loss with ordinal regularization. Across 5 levels of DR severity, we report mean quadratic kappa of 0.802 ±0.010 and AUC of 0.831 ±0.018, and across 3 categories of DME risk, we report kappa of 0.759 ±0.067 and AUC of 0.895 ±0.026. Our findings support the feasibility of joint transformer-based grading for DR and DME using a single computational architecture.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
L. Qiao, Y. Zhu and H. Zhou, “Diabetic Retinopathy Detection Using Prognosis of Microaneurysm and Early Diagnosis System for Non-Proliferative Diabetic Retinopathy Based on Deep Learning Algorithms,” in IEEE Access, vol. 8, pp. 104292-104302, 2020.
A. Bilal, L. Zhu, A. Deng, H. Lu and N. Wu, “AIbased automatic detection and classification of diabetic retinopathy using U-Net and deep learning,” Symmetry, vol. 14, no. 7, Art. no. 1427, 2022.
M. Chetoui and M. A. Akhloufi, “Explainable Diabetic Retinopathy using EfficientNET,” 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Montreal, QC, Canada, pp. 1966-1969, 2020.
Y. Miao and S. Tang, “Classification of diabetic retinopathy based on multiscale hybrid attention mechanism and residual algorithm,” Wireless Communications and Mobile Computing, vol. 2022, Art. no. 5441366, 2022.
S. Gayathri, V. P. Gopi and P. Palanisamy, “A lightweight CNN for diabetic retinopathy classification from fundus images,” Biomedical Signal Processing and Control, vol. 62, Art. no. 102115, 2020.
I. Kandel and M. Castelli, “Transfer learning with convolutional neural networks for diabetic retinopathy image classification: A review,” Applied Sciences, vol. 10, no. 6, Art. no. 2021, 2020.
A. Bora et al., “Predicting the risk of developing diabetic retinopathy using deep learning,” The Lancet Digital Health, vol. 3, no. 1, pp. e10–e19, Jan. 2021.
A. Bilal, G. Sun, Y. Li, S. Mazhar and A. Q. Khan, “Diabetic Retinopathy Detection and Classification Using Mixed Models for a Disease Grading Database,” in IEEE Access, vol. 9, pp. 23544-23553, 2021.
J. Cao, J. Chen, X. Zhang, Q. Yan and Y. Zhao, “Attentional mechanisms and improved residual networks for diabetic retinopathy severity classification,” The Journal of Healthcare Engineering, vol. 2022, Art. no. 9585344, 2022.
J. D. Bodapati, N. S. Shaik and V. Naralasetti, “Deep convolution feature aggregation: An application to diabetic retinopathy severity level prediction,” Signal, Image and Video Processing, vol. 15, no. 5, pp. 923–930, 2021.
M. Z. Atwany, A. H. Sahyoun and M. Yaqub, “Deep Learning Techniques for Diabetic Retinopathy Classification: A Survey,” in IEEE Access, vol. 10, pp. 28642-28655, 2022.
M. Nahiduzzaman, M. R. Islam and R. Hassan, “ChestX-Ray6: Prediction of multiple diseases including COVID-19 from chest X-ray images using convolutional neural network,” Expert Systems With Applications, vol. 211, Art. no. 118576, 2023.
M. M. Farag, M. Fouad and A. T. Abdel-Hamid, “Automatic Severity Classification of Diabetic Retinopathy Based on DenseNet and Convolutional Block Attention Module,” in IEEE Access, vol. 10, pp. 38299-38308, 2022.
C. Bhardwaj, S. Jain and M. Sood, “Transfer learning based robust automatic detection system for diabetic retinopathy grading,” Neural Computing & Applications , vol. 33, no. 20, pp. 13999–14019, 2021.
Z. Gu, Y. Li, Z. Wang, J. Kan, J. Shu and Q. Wang, “Classification of diabetic retinopathy severity in fundus images using the vision transformer and residual attention,” Computational Intelligence and Neuroscience, vol. 2023, Art. no. 1305583, 2023.
J. Cao et al., “Prediction of response to antivascular endothelial growth factor treatment in diabetic macular oedema using an optical coherence tomography-based machine learning method,” Acta Ophthalmol., vol. 99, no. 1, pp. e19–e27, Feb. 2021.
M. R. Islam et al., “Applying supervised contrastive learning for the detection of diabetic retinopathy and its severity levels from fundus images,” Computers in Biology and Medicine, vol. 146, Art. no. 105602, 2022.
C. Liu, W. Wang, J. Lian and W. Jiao, “Lesion classification and diabetic retinopathy grading by integrating softmax and pooling operators into vision transformer,” Frontiers in Public Health, vol. 12, Art. no. 1442114, 2024.
N. J. Mohan, R. Murugan, T. Goel and P. Roy, “ViT-DR: Vision Transformers in Diabetic Retinopathy Grading Using Fundus Images,” 2022 IEEE 10th Region 10 Humanitarian Technology Conference (R10-HTC), Hyderabad, India, pp. 167-172, 2022.
S. Vijayalakshmi, J. S. Manoharan, B. Nivetha and A. Sathiya, “Multi-task deep learning framework combining CNN: Vision transformers and PSO for accurate diabetic retinopathy diagnosis and lesion localization,” Scientific Reports, vol. 15, no. 1, Art. no. 35076, 2025.
W. Zhang, V. Belcheva and T. Ermakova, “Interpretable deep learning for diabetic retinopathy: A comparative study of CNN, ViT, and hybrid architectures,” Computers, vol. 14, no. 5, Art. no. 187, 2025.
R. Bhoopalan, P. Sekar, N. Nagaprasad, T. R. Mamo and R. Krishnaraj, “Task optimized vision transformer for diabetic retinopathy detection and classification in resource constrained early diagnosis settings,” Scientific Reports, vol. 15, no. 1, Art. no. 39047, 2025.
