Samsung working on standalone VR headset with eye/hand tracking & facial expression recognition

At last month’s Mobile World Congress Shanghai, Samsung showed a secret standalone virtual headset prototype to partners. As spotted on VR Focus, the product uses technologies allowing it to track eye and hand movement as well as determine various facial expressions.

Dubbed Exynos VR III, the head-mounted accessory is apparently a successor to another Samsung headset prototype, called Exynos VR II, that was never officially released. Samsung already offers a virtual reality headset in the form of the Gear VR device which requires the user to dock and undock their smartphone every time they use it.

Thanks to Visual Camp, a VR company that developed eye-tracking technology for the secret VR headset, we know it’s powered by a Samsung-designed 10nm hexa-core chip.

The chip includes a pair of Samsung M2 CPU cores clocked at 2.5 GHz, four ARM Cortex-A53 CPU cores clocked at 1.7 GHz and ARM Mali G71 MP20 graphics capable of driving two built-in 2,560-by-1,440 pixel displays at 90Hz or a single 4K external screen at 75Hz.

As mentioned, unlike Samsung’s current Gear VR headset that requires a smartphone to process data and render visuals, this all-in-one head-mounted display prototype packs in all the technology needed to render virtual worlds and apps standalone.

Visual Camp’s press release announcing the Samsung deal says its eye-tracking tech lets VR headsets conserve power by rendering parts of a scene the user is currently looking at very high resolution while showing anything in peripheral vision in reduced resolution.

This technique is known as “foveated rendering”.

“Several other technologies will be applied to the Exynos 3, in addition to the company’s eye-tracking technology, including hand tracking, voice recognition, and facial expression recognition,” reads the press release.

A measurement of the CPU power consumption of Samsung Electronics’ Exynos 8890 chip resulted in the relatively low average figure of less than three percent, said Visual Camp.

Companies like Apple, Google and Facebook are researching eye-tracking technology, too.

Apple is rumored to be working on a digital glasses or a virtual headset product that may use optics by German specialists Carl Zeiss, thought to be released in 2018 or 2019. The Cupertino giant recently acquired SensoMotoric Instruments for an undisclosed sum.

SensoMotoric Instruments is a German company that specializes in eye tracking. Their technology also uses foveated rendering, understands facial expressions and recognizes participant gestures and external events.