Development of Thai Sign Language Detection and Conversion System into Thai with Deep Learning
Article Sidebar
Main Article Content
Abstract
The hearing and speech impaired can communicate using sign language instead of word, which causes obstacles in communicating with unimpaired person. To help the hearing and speech impaired to communicate with the general public, this paper presents. It takes in key points of hands, faces and gestures, using the MediaPipe Framework to detect key points. The information was gathered from video gestures in Thai Sign Language to extract the coordinates of important points. The data were then divided into 80% of the training dataset and 20% of the test dataset. The data were used to train the model using the Long Short-Term Memory (LSTM) neural network algorithm for posture analysis. The model layers and experiment were modified by modifying the parameters. It was found that the model required node addition and class modification. This allowed the model to prevent underfitting problems and increased learning parameters. The accuracy was 0.83 after the results were in Thai words. It will be rendered in a mobile application developed with the Flutter Framework, which connects the model and the application with an API developed with the Flask API.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
References
Abdul, W., Alsulaiman, M., Amin, S.U., Faisal, M., Muhammad, G., Albogamy, F.R., Bencherif, M.A. and Ghaleb, H. (2021). Intelligent real-time Arabic sign language classification using attention-based inception and BiLSTM. Computers and Electrical Engineering 95: 107 - 395. doi: 10.1016/j.compeleceng.2021.107395.
Bora, J., Dehingia, S., Boruah, A., Chetia, A.A. and Gogoi, D. (2023). Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning. Procedia Computer Science 218: 1384 - 1393. doi: 10.1016/j.procs.2023.01.117.
Chaikaew, A., Somkuan, K. and Yuyen, T. (2021). Thai Sign Language Recognition: an Application of Deep Neural Network. In: 2021 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunication Engineering. IEEE, Cha-am. 128 - 131. doi: 10.1109/ECTIDAMTNCON51128.2021.9425711.
Chumchim, P. and Maneerat, P. (2020). Application development of sign language translation systems for the hearing-impaired people. PKRU SciTech Journal 4(1): 22 - 32.
Halder, A. and Tayade, A. (2021). Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning. International Journal of Research Publication and Reviews 2(5): 9 - 17.
Obi, Y., Claudio, K.S., Budiman, V.M., Achmad, S. and Kurniawan, A. (2023). Sign language recognition system for communicating to people with disabilities. Procedia Computer Science 216: 13 - 20. doi: 10.10 16/j.procs.2022.12.106.
Promrit, N. and Waijanya, S. (2019). Model for Practice Badminton Basic Skills by using Motion Posture Detection from Video Posture Embedding and One-Shot Learning Technique. In: Proceedings of the 2019 2nd Artificial Intelligence and CloudComputing Conference. Kobe, Japan. 117 – 124. doi: 10.1145/3375959.3375981.
Venugopalan, A. and Reghunadhan, R. (2021). Applying deep neural networks for the automatic recognition of sign language words: A communication aid to deaf agriculturists. Expert Systems with Applications 185: 115601. doi: 10.1016/j.eswa.2021.115601.
Sundar, B. and Bagyammal, T. (2022). American Sign Language Recognition for Alphabets Using MediaPipe and LSTM. Procedia Computer Science 215: 642 - 651. doi: 10.1016/j.procs.2022.12.066.
Zhang, Y., Min, Y. and Chen, X. (2021). Teaching Chinese Sign Language with a Smartphone. Virtual Reality & Intelligent Hardware 3(3): 248 - 260. doi: 10.1016/j.vrih.2021.05.004.