Helping the deaf: How a Pune startup is leveraging AI to give sign language voice

Parikshit Sohoni, co-founder of Glovatrix with the AI-enabled gloves that can translate sign language into voice

According to World Health Organization (WHO), almost 63 million Indians suffer from significant auditory impairment, or in other words, are deaf or hard of hearing. Communicating with them hinges on interpreters (through sign language), or takes place through writing, which is again not optimal.

To tackle this gap in communication, a Pune-based startup, Glovatrix, has developed gloves that leverage artificial intelligence to translate sign language into voice. For this purpose, the startup has been training its own algorithm based on datasets that it is building from scratch.

To train this unique dataset, Glovatrix, whose current employee count stands at three, including its two co-founders, records a sign language gesture 50 times, then tracks the data flowing from these 50 gestures and trains the AI on that data, said Parikshit Sohoni, co-founder of the startup.

The gloves, which are in their seventh iteration of development, have sensors embedded. The AI algorithm sits on the cloud. An app that the startup has developed works as a gateway between the AI algorithm and the sensors in the glove, Sohoni explained.

Moneycontrol was provided a live demonstration of the technology during the recent Startup Mahakumbh event in New Delhi. During the demonstration, Sohoni, who was wearing the gloves, said, "Hello, how are you?" in sign language, which was correctly translated into voice through the app.

However, that's not all that Glovatrix is aiming to do with their AI-enabled gloves. "What we are doing is essentially providing two way communication for the deaf individual. If I am deaf, I will do the sign language, the  gloves will convert that to speech. But whenever you say something, there will be microphones and a screen retrofitted to the gloves, which will be able to translate from voice to text or image," he said.

Going ahead, Glovatrix plans to employ 10 deaf people to record more gestures. "We will employ deaf people of various heights and genders to capture the variability of the same gesture," he explained.

The startup, which has been developing the product for the last three years, has secured funding of Rs 75 lakh through government grants and so on. It initially plans to position the device as a workplace communications tool.

"In Indian sign language, there are 20,000 gestures and we cannot incorporate them all. But when you look at a particular workplace, like for example, the retail or hospitality industry, if a person is working in a hotel, he has to interact with his colleagues, manager, clients, customers. So, around 400-500 gestures are enough for them to communicate in that particular workplace. So, we'll just record those gestures, and we'll deploy."

Going ahead Sohoni plans to spruce up the gloves. "If you see right now, it looks like we have stitched all these things together with our own hands, and there is a generous use of glue everywhere to put it together. The look and products need to be better."

Glovatrix was founded in 2020 by Sohoni, who earlier used to work as a data scientist, and Aishwarya Karnataki an electrical engineer.