Hearing-impaired teacher tests smart glasses that read lips

As a baby, Yao Dengfeng lost almost all of his hearing. Growing up, he felt that he would live in a silent world for the rest of his life, as he could not talk and interact the same way as other children.

In 1984, the medical field was not very developed for those with aural issues, and Yao's doctor said that his hearing would never recover. However, Yao's mother did not give up.

"I still remember my mother continued to train and guide me to speak out, and she once spent half of her annual wages on a speech training device. After countless attempts, I finally learned to produce sounds, although I could not hear," Yao says.

Though Yao's mother kept encouraging him to talk with others, it became even harder when he attended primary school, since a large part of his understanding in class depended on guessing. "If there had been a device helping me 'see' what the teacher said, it would have been much easier," he says.

At the Special Education College of Beijing Union University, Yao, now a 42-year-old associate professor, researches computer science, linguistics and education, trying to enable more hearing-impaired people to interact with the world.

In an audiovisual information cognitive computing laboratory, he is testing a special pair of glasses that can transcribe voice into text on the lenses, through which hearing-impaired people can actually "see" what others say rather than guess through lip-reading.

Yao delivered a 40-minute speech in English about information accessibility at the International World Wide Web Conference in Spain in 2009. It took him about 20 years from understanding to expressing.

"I hope there will be a barrier-free society in which each of us can equally obtain and use information," he says.

Currently, Yao concentrates on the machine recognition of sign language, through which hearing-impaired people can communicate with others more conveniently. "My first graduate students studied the machine recognition of traffic police gestures, which has been applied to autonomous vehicles. Some dangerous acts, smoking at a gas station, for example, can also be detected to avoid potential risks," he says.

"China is now progressing toward a barrier-free society in which information is accessible to the hearing-impaired," Yao says, adding that a sign-language database can greatly facilitate recognition, but more video data is needed to enhance its scope and preciseness.

Yao realized that the study at its present stage is still in the phase of visual signal recognition, but speech identification is even more important for helping the hearing-impaired to acquire information and communicate with others.

"An interdisciplinary study, covering linguistics, computer science and brain cognition, is required to further improve the information accessibility for the hearing-impaired, which is also my research orientation, and I feel a sense of mission," Yao says.

Nowadays, society offers more care and help to disabled people, as they also strive to embrace a bright future. "Many of my students who are hearing-impaired endeavor to practice what they have learned to change their own life with great confidence and perseverance," he says.

When Yao applied for a master's degree at Peking University, he asked the professors before the interview whether they could write down the questions for him in case he failed to hear the words clearly.

"Actually, I was able to communicate with others through lipreading at that time, but for the people that I first met, I had to gradually get used to their pronunciation. It was the smile of the interviewers that made me relaxed and assured," he says.

Now, Yao, an associate professor with a master's degree from Peking University and a doctorate from Tsinghua University, brings knowledge and hope to his students, inspiring hearing-impaired people like him to tackle the difficulties and pursue better lives.

"I believe that the strength of willingness and knowledge can vitalize the world, which can lead us to overcome the obstacles in our life. I hope more people can join us in researching information accessibility, in order to allow more people with hearing impairment to communicate with others easily," Yao says.

"We should endeavor to assist more people with disabilities, as well as the elderly, to live in an accessible world, enabling more people to find the value of life," he adds.

Xinhua