Optimized communication system for sign language speakers

Author: Guram guram
Keywords: Sign language, Sign language recognition, Keypoint detection, Wireless Communication, Three-dimensional animation
Annotation:

The work echoes the issue that modern communication systems should be adapted to persons with disabilities. This article is about creating and studying of the optimized communication system for people with hearing impairment and muteness. The system proposes replacing of the standard video call with animated messages. Deaf people will have the opportunity to establish continuous communication while using less communication resources, which will reduce their expenses. The main essence of the innovative communication system is to optimize the information for transmission. In order to make the communication efficient, it is better to transmit not images, but only necessary information. By using specific algorithms, we can observe that part of image, where are shown hands and determine their position. As a result, devices will exchange not full image, but the coded information relevant to signs. After receiving the coded information, the device will display an animated object on the screen. Specifically, there will be shown the so-called avatar, which will display signs relevant to coded information. As a result, a deaf person will receive the information sent by another person. The purpose of the project is to study the effectiveness of the described innovative method. This includes the following tasks: Building a prototype of the communication system, testing and optimizing if necessary. Most importantly, the effectiveness of the system should be tested by people with disabilities. At this point, there has been held research of the system for getting information about signs and the three-dimensional animated model. Two different approaches were discussed for getting information about signs from images: Sign language recognition and keypoint detection. The ready-made systems have been found and tested for each of them. It has been concluded that it would be better for the communication system to use the keypoint detection method. Animation model control software and ways of their integration into the project were also discussed.

Lecture files:

The optimized communication system for sign language speakers [en]
ოპტიმიზირებული საკომუნიკაციო სისტემა ჟესტების ენით მოსაუბრეთათვის [ka]