University students create glove to interpret sign language with AI

Handy’s story began in July of last year, when a group of six university students who did not know each other had to work as a group as part of the Tech4Good contest, an event organized annually in the context of the Seeds for the future program, which seeks to train young people in technological topics.

The idea of ​​the contest is that during the week that the program lasts, the groups develop a technological project that is in line with the Sustainable Development Goals. During the first two days, the group to which Javiera O’Ryan belonged did not know what to propose, until the International Business student at the University of Los Andes had an idea by looking at her Instagram.

“I follow a mother who has two deaf children and I thought that it is really a big problem and that it had to be super complicated to communicate with them and what would happen if they did not have cochlear implants, which are not universal for everyone,” recalls Javiera O’Ryan, currently 23 years old.

“Based on this, I posed the problem to my group, thinking that “Deaf people cannot do things as basic as making an emergency call or going to do a procedure at the Civil Registry, or going to a hospital where they just don’t have an interpreter.”

The idea for Handy arose from the desire to help those who use sign language to communicate better with the rest of society.

From that they began to think about solutions that would allow deaf people to communicate through sign language with people who do not use those codes.

They went from cameras to mini drones, until they came up with the idea of ​​gloves that were capable of measuring movement.

“We believed it was the most inclusive because it was not something so out of the ordinary or ostentatious,” says O’Ryan.

With that idea in mind, Handy was born, a glove that uses Artificial Intelligence to interpret sign language patterns through an app.

Thus, the cell phone allows if the person is mute to reproduce what they say from a voice or, conversely, to process an external voice and channel it through an avatar in sign language.

“That is important, because people who speak in sign language have a totally different grammar than we have in the spoken language. The grammar of sign language works in a much more spatial way and even the sentences are structured in a different way,” explains Javiera O’Ryan.

“So, being able to connect an audio with an avatar in sign language makes communication much more fluid, because it is as if a deaf person were talking to another deaf person.”

In addition, the application includes a section that functions as a social network. “Users create accounts and can interact with each other because, although the deaf community is very close-knit, it is a fairly closed community. It is difficult for them to go out and interact with others. So this app will open up the world a little to them and give them more opportunities to socialize,” says the student.

This is the digital prototype that Handy created, which explains the functions that the gloves will have.

This proposal at a theoretical level made them winners of Tech4Good to O’Ryan and his group, made up of Camilo Araya, student of Civil Engineering at the University of Chile; Francisco Zúñiga, civil engineering student at the Federico Santa María University; Rodolfo Fuentes, Computer Engineering student at Inacap; Krishna Amigo, Law student at Usach, and María Fernanda Araya, Commercial Engineering student at Adolfo Ibañez University.

Furthermore, the project made them the first Chileans to reach the Tech4Good global final, which took place in January of this year, in which they received mentoring and were able to further investigate the market for their solution, which led them to give their proposal a small twist.

“We were going to design this for adults, but another window opened, which was to design it for children as well. The same interpreters and deaf people told us that as adults it is difficult to begin to adapt to new things. This opened the window for us to be able to develop it in preschool children and to be able to use the same technology that Handy has to be able to teach from that age, which is where you can get the most benefit from it,” says O’Ryan.

In parallel with his participation in the different instances of Tech4Good, Handy was also in Voices, a contest organized since 2022 by Credicorp that seeks to reward innovative ideas from young people.

There they were selected to travel to the regional final of the event in Peru, which took place at the end of last year and where they won first place.

The project won both the jury and the public vote, obtaining US$15,000 and a place to participate in the One Young World Summit 2024 Montreal, the most important meeting of young leaders in the world, to which O’Ryan will travel in September as a representative of Handy.

a27046b963.jpg
Javiera O’Ryan, one of the creators of Handy, is also an ATA Taekwondo world champion.

According to Javiera O’Ryan, these instances have greatly nourished the project. “They always tell us to focus on the problem and not the solution. So some of the group took sign language courses to really understand how it is spoken and we have also interviewed people who are interpreters to see what they think about the idea,”

In addition, the financial prize allowed them to begin the prototyping phase of Handy. “The idea in the short term is to finish the prototype to be able to test it in the market, and in the long term, the goal as a group is to be able to spread this to the masses in Chile, from there in Latin America and hopefully reach the rest of the world,” he says.

The student also won a personal award in Seed for the futureranking within the top 5% of program participants globally. That guaranteed him a place on a trip to China that he will take in June to delve even deeper into the technology industry.

O’Ryan combines his student activity and his work at Handy with sports, since She is a national ATA Taekwondo team, a discipline in which she has been a four-time world champion. For her, this passion has also been useful in her foray into the world of innovation.

“I believe that sports instill in us values ​​of perseverance, of moving forward and also of managing time, which is very necessary to be able to stand out. And although one is competitive, the person one competes with the most is oneself. That goes hand in hand with technology, especially the most valuable thing about always moving forward,” he closes.

 
For Latest Updates Follow us on Google News
 

-

PREV Cryptocurrencies today: price of the main digital currencies as of June 16
NEXT designed in Italy for the world