• Subscribe

Breaking the silence of deaf people

Image of alphabet of the American Sign Language
The MMS Sign tool compares written text with the equivalent letters and words in sign language, such as American Sign Language above, and displays the message via a 3D avatar character. Image courtesy Wikimedia Commons.

In Tunisia, researchers are developing easy-to-use web tools to automatically translate written text into sign language.

In developing countries, the majority of deaf people are illiterate and cannot use mobile phone Short Message Service (SMS) or text messages. Their preferred language is sign language according to a research paper by scientists from the University of Tunis.

The WebSign project aims to break the silence of deaf people. "It's an initiative to help deaf individuals use Information and Communications Technology (ICT) and hearing people to communicate with deaf ones. The project is really exciting, especially when meeting and getting in touch with the target community," said Mohamed Jemni, head of the research laboratory at the University of Tunis and director of the Al Khawarizmi Computing Center.

One application they've already released is called MMS Sign, a free service for deaf people who want to use mobile phones, but cannot read. "There are no studies in the literature concerning projects that handle deaf and hard of hearing issues," said Jemni.

The application automatically translates an SMS sent from a mobile phone. It converts it into a Multimedia Messaging Service (MMS) that displays a 3D Computer-Generated-Image (CGI) avatar who acts-out the message in sign language.

This is how it works: if someone sends a message to a deaf person, e.g. "I will be late", this is sent to a telecom operator on the mobile phone. Once this text message is received by the telecom operator, the MMS Sign application converts the SMS into an MMS which is automatically sent to the deaf person. The MMS received by the deaf person contains the sign message, "I will be late".

"For many deaf people, this is the only way to receive messages and use mobile technology to communicate. This is very important for emergencies, asking for help, or communicating with parents. This service is already available in Tunisia. More than 200 deaf individuals were involved in the testing phase. More than 1,000 users use our products in Tunisia and some Arabic countries," said Jemni.

Do you like to eat green apples?

The MMS Sign tool requires access to a whole host of resources to manage millions of words. Jemni said, "WebSign needs huge computational resources and also sign language experts, deaf users, and linguists to enrich sign dictionaries."

The readable-text-to-sign-language translation is done through a machine learning approach. These are algorithms that compare the collection of words within the English language (also known as a corpus), in machine-readable form, with sign language, creating a parallel corpora of languages.

Image of MMS Sign application 3D avatar character.
The 3D avatar character animation is helping around 1,000 illiterate deaf people in Tunisia and other arabic countries. Image courtesy Mohamed Jemni.

A mathematical algorithm takes a given sentence in English and finds the most appropriate translation in sign language. An example of a word string would be: "Do you like to eat green apples?" with the equivalent sign translation being "APPLE, GREEN, YOU LIKE EAT?"

The data volume is enormous; the English language corpus alone has 52 million words. Jemni and his colleagues are using the scientific computing resources of the EUMEDGrid-Support e-infrastructure to greatly reduce the processing time and improve the quality of translations. EUMEDGrid is an e-infrastructure network that supports 25 sites across 13 countries.

"We opted to use the EUMEDGrid infrastructure in order to generate a very big parallel corpus of sign language - the biggest parallel corpus of sign language in the world - to improve the efficiency of the automatic translation. EUMEDGrid reduces processing times from several months to several minutes," said Jemni.

The WebSign project was presented by Jemni at the Mediterranean e-Infrastructures (EUMED4) conference in Amman, Jordan, on 13 December 2011, in conjunction with the First International Platform on Integrating Arab e-Infrastructure in a Global Environment.

Now, Jemni and his team are working on integrating the WebSign project with the EUMEDGrid science gateway. A science gateway is a pre-configured set of tools, applications, and data integrated into a single portal and accessible via a web browser. More applications are being developed. These include a screen reader that helps deaf people to sign what's written on their screen, and a sign language course to teach deaf and hearing people sign language.

Eventually, the project will be transferred to a cloud service. Jemni said, "once deployed as a cloud, anyone could access it and translate a written text into sign language regardless of the device, software, or community they belong to."

"To my knowledge, it is the only example of its kind in the grid world. We are currently brainstorming about how to integrate WebSign in the EUMEDGrid science gateway," said Roberto Barbera, a senior researcher at the University of Catania.

Join the conversation

Do you have story ideas or something to contribute? Let us know!

Copyright © 2023 Science Node ™  |  Privacy Notice  |  Sitemap

Disclaimer: While Science Node ™ does its best to provide complete and up-to-date information, it does not warrant that the information is error-free and disclaims all liability with respect to results from the use of the information.


We encourage you to republish this article online and in print, it’s free under our creative commons attribution license, but please follow some simple guidelines:
  1. You have to credit our authors.
  2. You have to credit ScienceNode.org — where possible include our logo with a link back to the original article.
  3. You can simply run the first few lines of the article and then add: “Read the full article on ScienceNode.org” containing a link back to the original article.
  4. The easiest way to get the article on your site is to embed the code below.