Google’s Project Tango: phone that maps 3D environments, senses depth and geometry

Google Project Tango (image 002)

Imagine directions to a new location not stopping at the street address, or never getting lost in a new building again. Or, how about playing hide-and-seek in your house with your favorite game character or competing against a friend “for control over physical space with your own miniature army”?

That’s the promise behind Project Tango, a new Google initiative which seeks to give mobile devices “a human-scale understanding of space and motion”.

The brainchild of Google’s Advanced Technology and Projects group, Project Tango is, on the surface, an unimpressive Android phone with a five-inch display and a four-megapixel camera.

But scratch deeper and you’ll find custom hardware like two Computer Vision Processors, sensors for measuring geometry and seeing in three dimensions, a camera which tracks motion and other goodies that basically turn this handset into a device that can map your environments, track its motion in full 3D space and much more…

Early prototypes of the device, of course, run Android and include development APIs to provide position, orientation and depth data to standard Android apps written in Java, C/C++, as well as the Unity Game Engine.

Check out the promo clip.

Project Tango is the result of a decade’s worth of research in robotics and computer vision across universities, research labs and Google’s industrial partners.

The initiative involves Johnny Chung Lee, a computer scientist and Human-Computer Interaction researcher at Google, who is perhaps best known for his work on Kinect development and extending the functionality of the Wii Remote controller.

Google Project Tango (image 004)

Here’s Google’s pitch:

What if you could capture the dimensions of your home simply by walking around with your phone before you went furniture shopping? What if directions to a new location didn’t stop at the street address? What if you never again found yourself lost in a new building?

What if the visually-impaired could navigate unassisted in unfamiliar indoor places? What if you could search for a product and see where the exact shelf is located in a super-store?

And this is what the prototype device looks like:

Google Project Tango (image 001)

The sensors track the full 3D motion of the device while simultaneously creating a map of the environment, according to Google:

These sensors allow the phone to make over a quarter million 3D measurements every second, updating it’s position and orientation in real-time, combining that data into a single 3D model of the space around you.

Project Tango is an experimental device so don’t expect it to be sold to end users anytime soon. “Project Tango is a focused exploration of what might be possible in a mobile platform,” explains the firm. “It is not part of Android today”.

“Today” here is key, I’d say.

Google Project Tango (image 003)

However, Google filed-testing Project Tango with cherry-picked developers indicates that the company is looking to add 3D depth sensing and environment mapping to mobile devices as soon as the technology is ready for prime time.

Project Tango is available to a limited number of developers: initially, Google will hand out only 200 devices which are expected to ship by March 14.

Here’s how 3D environment mapping works – it’s quite fascinating!

To learn more and apply for a development kit, head over to Google’s official Project Tango website.

I certainly would love my next iPhone to exhibit awareness of 3D space and motion.