Invention The Program of Hands Motion Detector for Translating Disabilities People's Sign Language

Abstract

The number of people with disabilities is currently increasing, and in Thai society, the proportion of individuals with disabilities is notably high, approximately 3% of the population (Krungthep Turakij, 2022). Providing support and facilitating accessibility for people with disabilities is of great importance. As part of these efforts, we developed a motion detection program aimed at translating sign language for individuals with disabilities. This program utilizes object detection techniques, such as MediaPipe, in combination with machine learning. It accesses the camera to capture sign language gestures and compares them with models trained on a sign language gesture dataset. The program then displays the corresponding meaning on the screen, with the goal of aiding communication for people with disabilities. The results indicate that the model performs well when tested with a benchmark dataset of 75,000 sign language gestures. However, challenges arise when gestures are unclear, incorrect, not included in the dataset, or closely resemble other gestures, which may lead to misclassification. Additionally, limitations due to insufficiently powerful training equipment have caused a delay in processing and displaying gesture meanings, with a lag time of approximately 5-10 seconds. Despite these challenges, the model achieves an accuracy of 76.40%, which is considered satisfactory. The program is also capable of translating the detected gestures into the Thai language.