A camera that lets people 'read' John Markoff

Updated: 2013-07-21 08:25

(The New York Times)

  Print Mail Large Medium  Small 分享按钮 0

A camera that lets people 'read' John Markoff

A camera that lets people 'read' John Markoff

JERUSALEM - Liat Negrin, an Israeli who has been visually impaired since childhood, walked into a grocery store here recently, picked up a can of vegetables and easily read its label using a simple and unobtrusive camera attached to her glasses.

Ms. Negrin, who has coloboma, a birth defect that perforates a structure of the eye and afflicts about 1 in 10,000 people, is an employee at OrCam, an Israeli start-up that has developed a camera-based system intended to give the visually impaired the ability to both "read" easily and move freely.

The OrCam device is a small camera worn in the style of Google Glass, connected by a thin cable to a portable computer designed to fit in the wearer's pocket. The system clips on to the wearer's glasses with a small magnet and uses a bone-conduction speaker to offer clear speech as it reads aloud the words or object pointed to by the user.

To recognize an object or text, the wearer simply points at it with his or her finger, and the device then interprets the scene.

The system recognizes a pre-stored set of objects and allows the user to add to its library - for example, text on a label or billboard, or a stop light or street sign - by simply waving his or her hand, or the object, in the camera's field of view.

Until now reading aids for the visually impaired and the blind have been cumbersome devices that recognize text in restricted environments, or, more recently, have been software applications on smartphones that have limited capabilities.

The system is designed to both recognize and speak "text in the wild," a term used to describe newspaper articles as well as bus numbers, and objects as diverse as landmarks, traffic lights and the faces of friends. It recognizes English-language text.

The device sells through the company's Web site for $2,500, about the cost of a midrange hearing aid. It is different from other technologies that have been developed to give some vision to people who are blind, like the artificial retina system called Argus II, made by Second Sight Medical Products. That system, which was approved by the United States Food and Drug Administration in February, allows visual signals to bypass a damaged retina and be transmitted to the brain.

The OrCam device is also drastically different from Google Glass, which also offers the wearer a camera but is designed for people with normal vision and has limited visual recognition and local computing power.

OrCam was founded several years ago by Amnon Shashua, a researcher who is a computer science professor at Hebrew University. It is based on computer vision algorithms that he has pioneered with another faculty member, Shai Shalev-Shwartz, and one of his former graduate students, Yonatan Wexler.

The advance is the result of both rapidly improving computers that can now be carried in a wearer's pocket and the computer vision algorithm developed by the scientists. The OrCam system is also representative of improvements in vision systems that use artificial intelligence.

OrCam's technique, known as Shareboost, is distinguished by the fact that as the number of objects it needs to recognize grows, the system minimizes the amount of additional computer power required.

"The challenges are huge," said Dr. Wexler, vice president of research and development at OrCam. "People who have low vision will continue to have low vision, but we want to harness computer science to help them."

One of the key challenges, Dr. Shashua said, was allowing quick optical character recognition in a variety of lighting conditions as well as on flexible surfaces.

"The professional optical character readers today will work very well when the image is good, but we have additional challenges - we must read text on flexible surfaces like a hand-held newspaper," he said.

Although the system is usable by the blind, OrCam is initially planning to sell the device to people in the United States who are visually impaired, which means that their vision cannot be adequately corrected with glasses.

OrCam said that worldwide there were 342 million adults with significant visual impairment, and that 52 million of them had middle-class incomes.

Tomaso Poggio, a computer scientist at the Massachusetts Institute of Technology with whom Dr. Shashua studied, is impressed by the OrCam device. "What is remarkable is that the device learns from the user to recognize a new product," he said. "This is more complex than it appears, and, as an expert, I find it really impressive."

The New York Times

(China Daily 07/21/2013 page11)