Soon it might be possible to access information on anything at all without having to lift a finger. Frannie Guan finds out about the incredible world of augmented reality.

Augmented reality takes another step forward inside the System and Media Lab (SyMLab) at Hong Kong University of Science and Technology (HKUST), where students seem to be playing with objects floating in midair, invisible to all but themselves.

It's like being Tony Stark (Robert Downey Jr) in the Iron Man movie franchise. Once he puts on his Iron Man suit, it starts feeding him information about his surroundings - the things he's looking at. He can access a wealth of information with only a wave of the hand, or even an extended gaze.

In the SyMLab at HKUST, a student wearing Google Glass seemed to be trying to grab something out of thin air. He watched a computer generate the image of a file which he could grasp between his fingers, and drop into a printer a few feet away to get a hard copy in a matter of seconds.

Wearing Google Glass, he still sees the real world, but the device can project virtual images onto the lens. He can look at his computer screen, while Google Glass, with its built-in camera, scans the computer and connects him within seconds. Personal Document Files (PDF) on the computer desktop are projected onto the lens, allowing him to manipulate them, dropping the virtual file to be printed in a real printer.

Frank Cheng Haofei, 23, a student working on the labs' augmented reality project, says it takes less time to print a file this way, than the conventional method of setting up print.

The system raises the level of interaction between the digital and physical worlds. Augmented reality combines the capabilities of various computer-related devices, smart phones, cameras, and so on, in devices like Google Glass, or Microsoft's HoloLens, to create what developers call "enhanced reality". The user needs only to stand looking at a restaurant and at the same time use augmented reality devices to go online and check the menu and restaurant reviews.

Cheng and his partner designed interconnected computers, a printer and a projection screen, all linked to the same network as Google Glass. The camera on the optical device is able to detect and identify the other devices, which create virtual content, like the file dropped into the printer, and project them on the lens of the glass. Users can manipulate the virtual content so that it becomes functional. Apart from printing files, users can transfer documents from one computer to another or project images onto a viewing screen.

"The idea of augmented reality is adding digital objects so that people can better interact with the physical world or get more information about it," said Hui Pan, assistant professor at HKUST's department of Computer Science and Engineering and director of SyMLab.

Writing on thin air

Augmented reality could be applied to automobile repair. By putting on the headset, the user can look under the hood of a car, view the internal structure, and get step-by-step instructions on how to replace automotive components. The instructions are overlaid on the user's view of the actual component needing repair.

The Royal Ontario Museum in Toronto adopted augmented reality so that visitors reading interpretive panels on the museum's dinosaur exhibits can see 3D videos with notes on the now-extinct species.

By advancing such evolutionary eyewear, Google and Microsoft are trying to achieve closer interaction between users, the real world and the virtual environment, a step beyond passive media like television.

Users of Microsoft's headset, HoloLens, which goes on sale early next year, will be able to work with a virtual, floating computer desktop, overlaid onto their real world surroundings. HoloLens is expected to respond to voice commands, hand gestures, even gazing at a particular point. With more developers jumping into the field of wearable and mobile augmented reality devices, Hui says the technology is set to play a significant role in the field for many years to come.

Cheng and his project partner, 21-year-old Lin Sikun, are considering developing their own mobile devices to perform the same tasks as Google Glass, taking photos, for instance. At the moment, however, their work is more focused on system design.

He pointed out Google Glass is unable to mark a hand gesture when made against a background similar to the user's skin color - a lacuna he and his partner hope to overcome some day. "We are providing references for other researchers and developers and much more can be done," said Cheng.

Hui added the system Cheng and Lin designed drew the attention of Deutsche Telekom, a leading European telecommunications provider. The company provides funding for the SyMLab at HKUST. The telecom giant is talking with some robotic companies in Germany and has started research into expanding the system so that it can be implanted in robots and other Internet of Things devices, Hui added.

Sharing 3D videos

While augmented reality developers are trying to bridge the gap between virtual and real-world elements, in another emerging technology, virtual reality, innovators are generating new ways to sharpen people's perspectives.

Pokin Yeung, a Hong Kong-born entrepreneur, co-founded a startup in San Francisco with 32-year-old Nick Scheri, after years of working in the video game industry, the field that first adopted virtual reality. The focus is on live streaming video on the Internet.

"Currently, the 3D videos people can see online are made by others. We are producing a camera that allows people to make their own 3D videos and share them with friends in real time," said Yeung.

Their device, called Peer, looks like a small robot, about 30 centimeters tall. It's equipped with two cameras in parallel so that their lenses are as far apart from each other as the average pair of human eyes. As a result, the two cameras shoot separate videos for both the left eye and the right eye. The two sets of videos then are presented on a mobile phone screen's left and right halves. After placing the mobile phone inside the 3D headsets containing two glasses allowing each set of videos visible only to each corresponding eye, viewers feel the effect of watching 3D.

"This kind of technology can allow people to be as close as possible to a place without actually being there," said Yeung. Scheri said he had invited his father, who lives in Tennessee, to be the first "customer", because his sister living in Oregon is expecting a baby next month. His father, without needing to travel to Oregon, can watch a video featuring his newborn grandchild both live and in 3D.

Friends living far apart from each other are now able to experience the feeling of watching movies sitting side by side. "After putting on the virtual reality headset, we are watching the same movie, and if I turn left, I can even talk to him," said Scheri.

Yeung believes the timing is perfect for their products, as virtual reality headsets such as Sony PlayStation VR, Samsung Gear VR and Oculus Rift will soon be on the market. Oculus Rift was purchased by Facebook for $2 billion last year. Facebook founder and CEO Mark Zuckerberg predicted virtual reality will follow mobile technology as the next major technological advance.

Peer was included in a development acceleration program sponsored by Infiniti, Nissan Motor Co. Ltd's luxury car brand, here in Hong Kong, Infiniti's global headquarters. The company believes Hong Kong has many advantages in terms of developing Internet of Things. "We feel this is the best place to build hardware since a lot of the factories are just across the border in Shenzhen," said Yeung. "And for day-to-day operations, we feel more stable here in Hong Kong."

Yeung's family moved from Hong Kong to Vancouver, Canada, when she was 8. After graduating from college, she relocated to San Francisco where she thought the startup environment was better. Yeung and Scheri are working with manufactures in Shenzhen. They hope to launch their product, Peer, next year.

Contact the writer at frannie@chinadailyhk.com

(HK Edition 12/08/2015 page10)