eBay’s HeadGaze technology responds to head motion using Apple’s TrueDepth camera

eBay has announced a new open source technology that could make it easier for people with motor impairments to shop or move around apps. HeadGaze uses Apple’s TrueDepth camera alongside Apple ARKit to allow you to navigate an app using your head motion. 

Created as part of an internship project at eBay, HeadGaze involves a virtual stylus that follows the motion of your head (up, down, side to side), and takes the head’s 3D information from ARKit and applies 3D geometry mapping to get the location of the “cursor” on the screen.

From here, interface widgets sense and respond to the “cursor” interaction. Similar to how a mouse navigates the cursor on a desktop, the design lets you point to any location on the screen with your head and activate designated “buttons.”

Take a look:

Muratcan Cicek, eBay Intern and Ph.D. Candidate at University of California, Santa Cruz explains:

HeadGaze enables you to scroll and interact on your phone with only subtle head movements. Think of all the ways that this could be brought to life. Tired of trying to scroll through a recipe on your phone screen with greasy fingers while cooking? Too messy to follow the how-to manual on your cell phone while you’re tinkering with the car engine under the hood? Too cold to remove your gloves to use your phone?

HeadGaze sounds like an exciting project, and because it’s open source, anyone can begin using it. The software is available through GitHub.com. Better still, the number of Apple devices with a TrueDepth camera is expected to increase considerably in the coming days. Now only available on the iPhone X, TrueDepth and Face ID are expected to expand to three new iPhones and two new iPad Pro tablets.

You can read more about HeadGaze at this link.