Leveraging Transfer Learning for Tri-Dhat Classification of Tongue Images in Traditional Thai Medicine
Article Sidebar
Main Article Content
Abstract
Traditional Thai medicine (TTM) is a popular and increasingly accepted treatment option. In TTM, tongue diagnosis is a highly efficient method for assessing overall health, yet its accuracy can vary significantly depending on the practitioner's expertise. In this work, we hypothesize that deep learning-based transfer learning (TL) methods can achieve high accuracy in the Tri-Dhat classification of tongue images, a system that aligns with TTM principles and categorizes the tongue into three types: Vata, Pitta, and Kapha. We propose an approach that uses raw pixel data and artificial intelligence (AI) to support TTM diagnoses. For our analysis, we used a unique dataset of genuine tongue images collected from our university's TTM hospital. To prepare the data, we performed class balancing and data augmentation. We then developed TL techniques with a variety of pretrained deep learning models. For performance comparisons, we utilized two-tailed paired t-tests and single-factor ANOVA. Our experiments showed that the DenseNet121 and Xception models produced the most significant results with cropped image datasets, including both DSLR- and mobile-taken images. Notably, an ensemble of these models yielded the highest average predictions. This ensemble achieved an accuracy of 0.96, a precision of 0.94, an F1 score of 0.96, a sensitivity of 0.96, and a specificity of 0.97. These results were further supported by a p-value of 0.0003 from the ANOVA analysis. We suggest that our methods could be effectively deployed in real-world scenarios to aid TTM practitioners in their diagnoses.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Z. Tian, D. Wang, X. Sun, Y. Fan, Y. Guan, N. Zhang, M. Zhou, X. Zeng, Y. Yuan, H. Bu, and H. Wang, “Current status and trends of artificial intelligence research on the four traditional Chinese medicine diagnostic methods: a scientometric study,” Ann Transl Med, vol. 11, no. 3, p. 145, Feb 2023.
M. S. Joshi, V. Umadevi, and A. B. N. Raj, “Computerized pragmatic assessment of Prakriti Dosha using tongue imagesPilot study,” Indian Journal of Science and Technology, vol. 13, no. 48, pp. 4679-4698, 2020. [Online].Available: https://doi.org/10.17485/IJST/v13i48.1626.
A. K. Sharma, Diagnostic Methods in Ayurveda. India: MLBD, 2008.
A. R. Jung, H.-Y. Lee, and M.-S. Hwang, “A Comparative Study on the Tongue Diagnosis between Korean medicine and Ayurveda,” The Journal of Korean Medicine, vol. 40, no. 2, pp. 63-71, 2019, published online: June 30, 2019. [Online]. Available: https://doi.org/ 10.13048/jkm.19017.
W. S. Kacera, Ayurvedic Tongue Diagnosis. Delhi, India: Motilal Banarsidass, 2007.
J. K. Anastasi, L. M. Currie, and G. H. Kim, “Understanding diagnostic reasoning in TCM practice: tongue diagnosis,” Altern Ther Health Med, vol. 15, no. 3, pp. 18-28, May-Jun 2009.
V. Tantiveerkul, Textbook of Traditional Thai Medicine, vol. 1-3. Bangkok: Traditional Medicine School, Wat Phra Chetuphon Vimolmangklararam Rajwaramahaviharn, 1957.
P. P. Prasatwet, Medical Studies: General Medicine, vol. 1. Bangkok: Samakkhi, 1986.
P. Sapcharoen, Compilation of Theory of Thai Traditional Medicine, Volume 1: The Canon of Diagnosis, vol. 1. Bangkok: Institute of Thai Traditional Medicine, 2537.
D. Miryala, P. Parvataneni, and G. Aliperi, “Computer aided image enhancement of tongue for diagnosis in ayurvedic medical treatment,” Applied Medical Informatics, vol. 34, no. 1, pp. 46-56, 2014.
M. H. Tania, K. Lwin, and M. A. Hossain, “Advances in automated tongue diagnosis techniques,” Integr Med Res, vol. 8, no. 1, pp. 42-56, Mar 2019.
X. Wang, J. Liu, C. Wu, J. Liu, Q. Li, Y. Chen, X. Wang, X. Chen, X. Pang, B. Chang, J. Lin, S. Zhao, Z. Li, Q. Deng, Y. Lu, D. Zhao, and J. Chen, “Artificial intelligence in tongue diagnosis: Using deep convolutional neural network for recognizing unhealthy tongue with tooth-mark,” Computational and Structural Biotechnology Journal, vol. 18, pp. 973-980, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2001037020300325.
L. Ding, X. you Zhang, D. yao Wu, and M. ling Liu, “Application of an extreme learning machine network with particle swarm optimization in syndrome classification of primary liver cancer,” Journal of Integrative Medicine, vol. 19, no. 5, pp. 395-407, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2095496421000650.
Y. Xiang, L. Shujin, C. Hongfang, W. Yinping, Y. Dawei, D. Zhou, and L. Zhiqing, “Artificial intelligence-based diagnosis of diabetes mellitus: Combining fundus photography with traditional chinese medicine diagnostic methodology,” BioMed Research International, vol. 2021, p. 5556057, 2021. [Online]. Available: https://doi.org/10.1155/2021/5556057.
J. Li, P. Yuan, X. Hu, J. Huang, L. Cui, J. Cui, X. Ma, T. Jiang, X. Yao, J. Li, Y. Shi, Z. Bi, Y. Wang, H. Fu, J. Wang, Y. Lin, C. Pai, X. Guo, C. Zhou, L. Tu, and J. Xu, “A tongue features fusion approach to predicting prediabetes and diabetes with machine learning,” Journal of Biomedical Informatics, vol. 115, p. 103693, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1532046421000228.
J. Li, J. Huang, T. Jiang, L. Tu, L. Cui, J. Cui, X. Ma, X. Yao, Y. Shi, S. Wang, Y. Wang, J. Liu, Y. Li, C. Zhou, X. Hu, and J. Xu, “A multistep approach for tongue image classification in patients with diabetes,” Computers in Biology and Medicine, vol. 149, p. 105935, 2022.
Y. Shi, H. Wang, X. Yao, J. Li, J. Liu, Y. Chen, L. Liu, and J. Xu, “Machine learning prediction models for different stages of nonsmall cell lung cancer based on tongue and tumor marker: a pilot study,” BMC Medical Informatics and Decision Making, vol. 23, no. 1, p. 197, 2023. [Online]. Available: https://doi.org/10.1186/s12911-023-02266-5.
X. Lu, H. Hu, W. Li, J. Deng, L. Chen, M. Cheng, H. Huang, W. Ke, W. Wang, and B. Sun, “Exploring hepatic fibrosis screening via deep learning analysis of tongue images,” Journal of Traditional and Complementary Medicine, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2225411024000294.
J. Han, M. Kamber, and J. Pei, Data mining: Concepts and Techniques, 3rd ed. MA, USA: Morgan Kaufmann, 2012.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” CoRR, vol. abs/1512.03385, 2015.
T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” CoRR, vol. abs/1603.02754, 2016. [Online]. Available: http://arxiv.org/abs/1603.02754.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in The Ninth International Conference on Learning Representations, Vienna, Austria, 4 May 2021. [Online]. Available: https://openreview.net/forum?id=YicbFdNTTy.
J. Yan, J. Cai, Z. Xu, R. Guo, W. Zhou, H. Yan, Z. Xu, and Y. Wang, “Tongue crack recognition using segmentation based deep learning,” Scientific Reports, vol. 13, no. 1, p. 511, 2023. [Online]. Available: https://doi.org/10.1038/s41598-022-27210-x.
K. He, G. Gkioxari, P. Doll´ar, and R. Girshick, “Mask R-CNN,” in 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988, Oct 2017.
L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation,” CoRR, vol. abs/1802.02611, 2018. [Online]. Available: http://arxiv.org/abs/1802.02611.
O. Ronneberger, P. Fischer, and T. Brox, “UNet: Convolutional Networks for Biomedical Image Segmentation,” in Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, N. Navab, J. Hornegger, W. M. Wells, and A. F. Frangi, Eds. Cham: Springer International Publishing, pp. 234-241, 2015.
Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation,” IEEE Transactions on Medical Imaging, vol. 39, no. 6, pp. 1856-1867, June 2020.
Y. Xue, T. Xu, H. Zhang, L. R. Long, and X. Huang, “SegAN: Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation,” Neuroinformatics, vol. 16, no. 3, pp. 383-392, 2018. [Online]. Available:https://doi.org/10.1007/s12021-018-9377-x.
L. Feng, W. Xiao, C. Wen, Q. Deng, J. Guo, and H. Song, “Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application,” J Vis Exp, no. 194, Apr 2023.
T. Yamane, Statistics : an introductory analysis, 2nd ed. New York: Harper and Row, 1967.
B. Prapaspong, S. Suwanphokhin, and U. Chaiklang, Royal Medical Literature: Medical Knowledge and Literary Heritage of the Nation. Bangkok: The Institute of the Thai Language, Department of Education, Ministry of Education, 1999.
J. L. Fleiss, “Measuring Nominal Scale Agreement Among Many Raters,” Psychological Bulletin, vol. 76, no. 5, pp. 378-382, 1971.
G. Lemaˆıtre, F. Nogueira, and C. K. Aridas, “Imbalanced-learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning,” Journal of Machine Learning Research, vol. 18, no. 17, pp. 1-5, 2017. [Online]. Available: http://jmlr.org/papers/v18/16-365.html.
K. Damkliang, T. Wongsirichot, and P. Thongsuksai, “TISSUE CLASSIFICATION FOR COLORECTAL CANCER UTILIZING TECHNIQUES OF DEEP LEARNING AND MACHINE LEARNING,” Biomedical Engineer455 ing: Applications, Basis and Communications, vol. 33, no. 03, p. 2150022, 2021. [Online]. Available: https://doi.org/10.4015/S1016237221500228.
K. Damkliang, T. Wongsirichot, C. Khongrak, and P. Suwannarat, “An Optimal Deep Learning Approach to BCa Tissue Detection using Case Studies,” ECTI Transactions on Computer and Information Technology (ECTI-CIT), vol. 17, no. 1, pp. 60-72, Feb. 2023. [Online]. Available: https://ph01.tci-thaijo.org/index.php/ecticit/article/view/250441.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09, Miami, FL, USA, pp. 248-255, June 2009.
G. Huang, Z. Liu, and K. Q. Weinberger, “Densely Connected Convolutional Networks,” CoRR, vol. abs/1608.06993, 2016.
M. Tan and Q. V. Le, “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” CoRR, vol. abs/1905.11946, 2019. [Online]. Available: http://arxiv.org/abs/1905.11946
C. Szegedy, S. Ioffe, and V. Vanhoucke, “Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” CoRR, vol. abs/1602.07261, 2016.
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” CoRR, vol. abs/1704.04861, 2017. [Online]. Available: http://arxiv.org/abs/1704.04861.
K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” CoRR, vol. abs/1409.1556, pp. 114, 2014.
F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800-1807, 2017.
K. Zhang, X. Zhang, and F. Ahmad, “Tongue image texture classification based on exception,” 9th International Conference on Computing and Pattern Recognition, pp. 261-266, 2020.
K. Damkliang, P. Thongsuksai, K. Kayasut, T. Wongsirichot, C. Jitsuwan, and T. Boonpipat, “Binary semantic segmentation for detection of prostate adenocarcinoma using an ensemble with attention and residual U-Net architectures,” PeerJ Computer Science, vol. 9, no. e1767, 2023. [Online]. Available: https://doi.org/10.7717/peerj-cs.1767.
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “HuggingFace’s Transformers: State-of-the-art Natural Language Processing,” https://huggingface.co/docs/transformers/index, 2020.
P. Lakubovskii, “Segmentation Models,” https://github.com/qubvel/segmentation_models, 2019.
K. Damkliang, J. Chumnaul, T. Sudkhaw, T. Yingtawee, and N. Saearma, “Multi-Model Approach for Tongue Image Classification in Traditional Thai Medicine,” International Journal of Online and Biomedical Engineering (iJOE), vol. 21, no. 5, pp. 1-16, 2025. [Online]. Available: https://doi.org/10.3991/ijoe.v21i05.53671.
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man´e, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Vi´egas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems,” https://www.tensorflow.org/, 2015, software available from tensorflow.org.
F. Chollet, J. J. Allaire, S. Pal, A. Gulli, R. Atienza, and S. Keydana, “Keras,” https://keras.io, 2015, [Online; accessed 16-November2023].
F. Morales, “vit-keras: Implementation of Vision Transformer (ViT) in Keras,” https://github.com/faustomorales/vit-keras, 2022.
