Dynamic Thai Sign Language Recognition using Recurrent Neural Network
Article Sidebar
Main Article Content
Abstract
Sign Language is a communication using a hand gesture that can pose on head to waist along with a facial emotion. There are numerous articles attempting to recognize dynamic sign language using machine learning. However, the dynamic sign language is a temporal continuous data. In addition, the positions of the hands and facial emotion are components that contribute to the completeness of sign language communication. Therefore, a sign language recognition methodology development is still challenge. This research aims to develop a Thai sign language recognition approach using recurrence neural network (RNN). The MediaPipe library applies to landmark extraction consisting of the hands, face and posture using by coordinate (x, y, z) totally 1,662 keypoint for RNN input. After that, these keypoints are learned by RNN technique consisting of long short-term Memory (LSTM) 2) gated recurrent unit (GRU) and 3) bi-direction LSTM (BiLSTM). The dataset consists of 10 words of Thai sign Language totally 1,000 videos that are established by volunteers sign language interpreters and hearing impaired. The experiment result demonstrates that an accuracy of the proposed method at 99% by LSTM and GRU.
Article Details
References
W. Daengrueang, J. Kaewsritong, and W. Intakan. "The Development of Thai Sign Language Multimedia on Home Economics for Students with Hearing Impairment." T he Golden Teak : Humanity and Social Science Journal, Vol. 28, No. 3, pp. 163 - 175, 2022.
P. Kaewdee, M. Koodduderm, and W. Kiewkam. "Structural Analysis of Occupational Vocabulary in Thai Sign Language for Sign Language Interpreters." Interdisciplinary Studies Journal, Vol. 23, No. 2, pp. 2 - 17, July - December, 2023.
R. Kanakala, J. Mohan, and K. Reddy. "Modelling a deep network using CNN and RNN for accident classification." Measurement: Sensors, Vol. 27, pp. 1 - 10, 2023.
M. Babaee, Z. Li, and G. Rigoll. "A dual CNN–RNN for multiple people tracking." Neurocomputing, Vol. 368, pp. 69 - 83, 2019.
C. Tatiyavoranun. "An Application of Artificial Neural Network for Thai Sign Language Recognition." Engineering Transactions, Vol. 23, No. 1, pp. 51 - 57, 2020.
A. Chaikaew, K. Somkuan, and T. Yuyen. "Thai Sign Language Recognition: An Application of Deep Neural Network." 2021 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical Computer and Telecommunication Engineering, Chiang Rai, pp. 128 - 131, 2021.
C. Damrongekarun, L. Pisitpipattana, S. Waijanya, and N. Promrit. "Development of Thai Sign Language Detection and Conversion System into Thai with Deep Learning." KKU SCIENCE JOURNAL, Vol. 51, No. 3, pp. 216 - 225, September - December, 2023.
G. H. Samaan, A. R. Widie, A. K. Attia, A. M. Asaad, A. E. Kamel, S. O. Slim, M. S. Abdallah, and Y. Cho. "MediaPipe's Landmarks with RNN for Dynamic Sign Language Recognition." Electronics, Vol. 11, No. 19, pp. 1 - 15, 2022.
M. Alnaggar, A. I. Siam, M. Handosa, T. Medhat, and M.Z. Rashad. "Video-based real-time monitoring for heart rate and respiration rate." Expert Systems with Applications, Vol. 225, pp. 1 - 11, 2023.
C. Bisogni, L. Cimmino, M. D. Marsico, F. Hao, and F. Narducci. " Emotion recognition at a distance: The robustness of machine learning based on hand-crafted facial features vs deep learning models." Image and Vision Computing, Vol. 136, pp. 1 - 15, 2023.
J. Bora, S. Dehingia, A. Boruah, A. A. Chetia, and D. Gogoi. "Real-time Assamese Sign Language Recognition using MediaPipe and Deep Learning." Procedia Computer Science, Vol. 218, pp. 1384 - 1393, 2023.
B. Sundar and T. Bagyammal. "American Sign Language Recognition for Alphabets Using MediaPipe and LSTM." Procedia Computer Science, Vol. 215, pp. 642 - 651, 2022.
S. Khumwongsa and W. Yawai. "Smart Application for Thai and English Sign Language Translation." Journal of Applied Informatics and Technology (JIT), Vol. 5, No. 2, pp. 178 - 194, 2023.
E. Gedkhaw. "The Performance of Thai Sign Language Recognition Using 2D Convolutional. Neural Networks." The 13th NPRU National Academic Conference, Nakhon Pathom University, Nakhon Pathom, Thailand, pp. 546 - 573, 2021.
K.E. A. Kumar, D. V. Kalaga, Ch. M. S. Kumar, M. Kawaji, and T. M. Brenza. "Forecasting of COVID-19 using deep layer Recurrent Neural Networks (RNNs) with Gated Recurrent Units (GRUs) and Long Short-Term Memory (LSTM) cells." Chaos, Solitons & Fractals, Vol. 146, pp, 1 - 12, 2021.
S. M. Al-Selwi, M. F. Hassan, S. J. Abdulkadir, A. Muneer, E. H. Sumiea, A. Alqushaibi, and M. G. Ragab. "RNN-LSTM: From applications to modeling techniques and beyond—Systematic review." Journal of King Saud University - Computer and Information Sciences, Vol. 36, No. 5, pp. 1 - 34, 2024.
S. Khan and V. Kumar. "A novel hybrid GRU-CNN and residual bias (RB) based RB-GRU-CNN models for prediction of PTB Diagnostic ECG time series data." Biomedical Signal Processing and Control, Vol. 94, pp. 1 - 18, 2024.
M. R. Ahmed, S. Islam, A.K.M. M. Islam, and S. Shatabda. "An ensemble 1D-CNN-LSTM-GRU model with data augmentation for speech emotion recognition." Expert Systems with Applications, Vol. 218, pp. 1 - 21, 2023.
W. Fan, J. Yao, S. Cui, Y. Wang, S. Xu, Y. Tan, F. Yang, and W. Wu. "Bi-LSTM/GRU-based anomaly diagnosis for virtual network function instance." Computer Networks, Vol. 249, pp. 1 - 16, 2024.
K. Subyen, W. Samhansub, and J. Suraseing. "Sign language processing software." RMUTSB Academic Journal, Vol. 4, No.1, pp. 46 - 56, 2016.
