Smart Application for Thai and English Sign Language Translation
Article Sidebar
Main Article Content
Abstract
Currently, the sign language is used as a means of communication for individuals with hearing impairments. Since most people do not have knowledge to use sign language, as a result, miscommunication may occur. The aim of this project is to develop a program for translating Thai and English sign language. It can translate Thai sign language, English sign language, Arabic numbers, and English words. MediaPipe algorithm is used for hand detection and find 21 Keypoints on each hand. This research is able to detect a maximum of two hands, allowing a total of 42 Keypoints to be used to make decisions for the classification of sign language, which encompasses a total of 68 symbols in the test. A total of 3 experimenters are assigned to experiment with the sign language recognition program 5 times per symbol and specified that the webcam must be approximately 50 centimeters away from the experimenter. As a result from the experiment, it was found that English sign language recognition was more accurate than Thai sign language. This is because some signs in Thai sign language require two levels of gestures. This causes errors in sign language recognition. This research has a sign language recognition result of 95.39%.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All authors need to complete copyright transfer to Journal of Applied Informatics and Technology prior to publication. For more details click this link: https://ph01.tci-thaijo.org/index.php/jait/copyrightlicense
References
Agency, A. (2021). Viral hand sign leads Spanish authorities to detain domestic abuser. Retrieved 2 March 2023. Retrieved from https://www.dailysabah.com/world/europe/viral-hand-sign-leads-spanish-authorities-to-detain-domestic-abuser
Camgo ̈z, N. C., Koller, O., Hadfield, S., & Bowden, R. (2020). Sign language transformers: Joint end-to-end sign language recognition and translation. In the Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, pp. 10023-10033. IEEE. https://doi.org/10.1109/CVPR42600.2020.01004
Chumchim, P. & Maneerat, P. (2020). Application development of sign language translation systems for the hearing-imparied people. PKRU SciTech Journal, 4(1), 22–32. https://ph01.tci-thaijo.org/index.php/pkruscitech/article/view/240583 [in Thai]
Daily News Online. (2021). September 23, International day of sign languages [23 ก.ย. วันภาษามือโลก]. Retrieved 21 February 2023. Retrieved from https://www.dailynews.co.th/articles/289869/ [in Thai]
Department of Empowerment of Persons with Disabilities. (2022). Report on the situation of persons with disabilities in Thailand [รายงานข้อมูลสถานการณ์ด้านคนพิการในประเทศไทย]. Retrieved 2 May 2023. Retrieved from https://dep.go.th/images/uploads/files/situation_sep65.pdf [in Thai]
Department of Provincial Administration. (2022). Population statistics from the civil registration (Monthly) [สถิติประชากรทางการทะเบียนราษฎร (รายเดือน)]. Retrieved 1 May 2023. Retrieved from https://stat.bora.dopa.go.th/stat/statnew/statMONTH/statmonth/#/view [in Thai]
Disabled World. (2022). Deaf communication: Sign language and assistive hearing devices. Retrieved 2 May 2023. Retrieved from https://www.disabled-world.com/disability/types/hearing/communication
Grishchenko, I. & Bazarevsky, V. (2022). MediaPipe holistic - Simultaneous face, hand and pose prediction, on device. Retrieved 25 February 2023. Retrieved from https://blog.research.google/2020/12/mediapipe-holistic-simultaneous-face.html?m=1
Jeekratok, K., Tongthaku, P., & Jampamoon, A. (2019). Thai sign language system in daily Life. Udon Thani Rajabhat University Journal of Science and Technology, 7(1), 101-113. https://ph01.tci-thaijo.org/index.php/scudru/article/view/181628 [in Thai]
Lapthanachai, N., Chomthong, A., Waijanya, S., & Promrit, N. (2023). Classification of nail abnormalities using convolutional neural network. Journal of Applied Informatics and Technology, 5(1), 18–35. https://doi.org/10.14456/jait.2023.2 [in Thai]
MediaPipe. (2023). Hand landmarks detection guide. Retrieved 20 February 2023. Retrieved from https://developers.google.com/mediapipe/solutions/vision/hand_landmarker
Nadee, C. & Yingkayun, K. (2020). Development of method for categorization on sign language picture using image plane adjustment technique. RMUTL Engineering, 5(1), 25–34. https://doi.org/10.14456/rmutlengj.2020.4 [in Thai]
Prajan, S. (2022). Thai fingerspelling in sign language [แบบสะกดนิ้วมือไทยในภาษามือ]. Retrieved 20 February 2023. Retrieved from https://www.gotoknow.org/posts/604277 [in Thai]
Saisangchan, U., Chamchong, R., & Suwannasa, A. (2022). Analysis of lime leaf disease using deep learning. Journal of Applied Informatics and Technology, 4(1), 71-86. https://doi.org/10.14456/jait.2022.6 [in Thai]
Sriboonruang, P., Tananchai, P., Khaggathog, K., Vijitkunsawat, W., & Anunvrapong, P. (2022). Software sign language translator to text and speech by using landmarks technique of MediaPipe. The Journal of Industrial Technology Suan Sunandha Rajabhat University, 10(2), 66–76. https://ph01.tci-thaijo.org/index.php/fit-ssru/article/view/249506 [in Thai]
Wadhawan, A. & Kumar, P. (2020). Deep learning-based sign language recognition system for static signs. Neural Computing and Applications, 32, 7957–7968. https://doi.org/10.1007/s00521-019-04691-y