At the end of my previous blog post, I gave a spoiler about the research topic we are currently working on with my classmate Aykhan. In this post, I will give detailed information about the project and explore it in terms of the evaluation criteria and questions taught during the classes.
What are we trying to do and What do we intend to achieve/demonstrate?
- Our project is about the detection of the Azerbaijani Sign Language. As we know, deaf and mute people face numerous difficulties communicating with society. In order to make their life easier, we are planning to generate software that will convert the sign language presented as a video into textual data.
So, we are trying to develop a kind of software or API that will convert continuous input video to text output.
Our aim is to allow Azerbaijani deaf-mute citizens to speak the “same language” with the whole nation.
What do we understand about the domain?
- At the moment, we are not much aware of the domain yet, but we are eager to learn Azerbaijani Sign Language at least. Already started to study the sign language alphabet.
What data do we think we will need?
- We ourselves are searching for proper and understandable data in various platforms, including Kaggle and etc. Several datasets are available in other languages, but not Azerbaijani. Therefore, we have to generate our video samples as well. Moreover, our supervisor Professor Jamaladdin Hasanov is in contact with Azerbaijani Governmental officials to find the appropriate data usable for our project.
These days, we are investigating the "RWTH-PHOENIX-Weather 2014" dataset which is publicly available.
What tools and methods do we propose to use?
- We are going to use OpenCV for the Computer Vision side, NLTK for the NLP side using Python. According to the initial research we did, CNN+LSTM is the best option for now. But we will try to find, use and test alternative approaches for Azerbaijani Sign Language Project.
Who cares? If you are successful, what difference will it make?
- All governmental and non-governmental services, also deaf-mute people care for this project. It will remove the communication barrier between the service sector and deaf-mute citizens.
In the end, I would like to conclude that although the idea has been realized for many Languages, there is still no Sign Language Detection for the Azerbaijani one. It will probably require much effort and time to work on it, but I personally believe the revolutionary effect of the results of this project (hopefully, if it is successful) on the Azerbaijani community. Additionally, we will study much through implementing and broaden our knowledge on Computer Vision, ML, and DL. So, this project is interesting to me because of both the self-study side and the social impact on the special part of the community.
That is all for now. I will continue writing blogs about the news of our project.
This is a great post. You have a possible dataset and a backup plan. Bravo! Picky comment about English -- I don't think this is "detection of Azerbaijani sign lunges", it is rather recognition or classification. (Detection problems are usually "yes/no, Azerbaijani sign language is present", instead of "what is being said right now").