How PrimeSense’s 3D sensing technology could improve Apple Maps

Apple Maps (Flyover, Disneyalnd, Paris)

Apple recently dropped a cool $350 million on the PrimeSense acqui-hire. An Apple spokeswoman has officially confirmed the deal last week, but refused to discuss her employer’s purpose or plans (as is the company’s wont). Unless you’ve been sleeping under a rock for the past few years, you must be familiar with PrimeSense, a Tel Aviv-based startup that licenses its 3D sensing hardware designs and chips to third-parties.

Perhaps the most famous PrimeSense licensee: Microsoft, whose Kinect motion-sensing system for the Xbox console is powered by PrimeSense.

For the sake of completeness, the much-improved Kinect that ships with the new Xbox One uses Microsoft’s own 3D sensing technology, not PrimeSense’s. Be that as it may, the PrimeSense deal has prompted a swirl of speculation among the Apple punditry. Some folks think Apple snapped up the Israeli startup for its rumored television set, others mention iPhones and iPads.

One well-informed source warns all of the speculated uses are too obvious and off mark. Instead, Apple needs PrimeSense to bolster up its Maps, the source insists…

The former Wall Street Journal reported Jessica Lessin has the scoop:

Industry sources tell me PrimeSense’s motion-sensing technology for gestural controls is a little bit behind and that Microsoft doesn’t use it in the Kinect.

But PrimeSense’s technology is much more strategic for mapping, according to one person familiar with the company.

She goes on to note that Matterport pays license fees to use PrimeSense technology and chips in its camera that maps objects in three dimensions.

primesense sensor

But how would PrimeSense improve Apple Maps?

We know Apple cares about mapping. The company bought WifiSLAM, an indoor GPS company, to help it map out malls and another indoor spaces in a race against Google, which is doing the same.

Sooner rather than later, our phones will pull up scans of real spaces we want to visit or may be approaching. Those two-dimensional maps will seem very obsolete.

In a way, Apple could (theoretically) utilize PrimeSense chips in future iPhones and iPads to crowd-source three-dimensional representation of users’ environments. Currently, 3D Flyover view in Apple Maps taps mountains of three-dimensional data collected by a fleet of expensive aircraft outfitted with high-resolution cameras.

Advanced algorithms (bought as part of Apple’s C3 Technologies acquisition) running on powerful machines crunch 2D aerial images, figuring out the third dimension. The process is time-consuming, expensive and complex in terms of logistics. If a future iPhone or iPad could optionally scan your surroundings and create an accurate 3D representation to help improve Apple’s mapping service, then I’m all for it.

Of course, Lessin’s suggestion does not exclude the use of PrimeSense chips in a future Apple TV hardware refresh. For what it’s worth, Lessin continues to hear that Apple “is more interested in set-top boxes” than full-on TV sets. Early Apple TV prototypes “haven’t had motion technology,” she writes, which isn’t preventing Apple from adding 3D sensing to the $99 set-top box over time.

I’m still not convinced people would prefer navigating the Apple TV interface by waving their hands, as opposed to using the Apple Remote or Apple’s free Remote app for the iPhone and iPad. Unless Apple invents a whole new 3D gestural world, I’m not expecting PrimeSense-enabled Apple TV (or iTV) anytime soon.

Xbox One (teaser 002)
The Xbox One and its much-improved Kinect sensor/


On a related note, Julie Clover of MacRumors has a nice take on how Apple might incorporate PrimeSense’s 3D technology into its products. In envisioning a few likely scenarios, her article covers interesting PrimeSense uses beyond television – from interactive gaming and indoor mapping to retail, 3D scanning and printing.

Hers is a well-researched article and a highly recommended read.

In a similar vein, The Loop’s Jim Dalrymple reminds us that object sensing tech has been around since the 1970s, when Patrick Winston first wrote about computer vision and described the artificial intelligence algorithms needed for a computer to distinguish the corners that make up a room.

Now that Apple owns PrimeSense, you can bet the company will mainstream 3D gestures in a big way, as much as it’s done with the iPhone’s Multi-Touch user interface, the iPhone 5s’s fingerprint scanning or the Mac’s point’n’click GUI.

“This stuff is consumer ready, about to make the leap from the niche gamer markets to the wider world around us,” he opines. “To me, this technology was made for the brains at Apple”.

The question is, are we as consumers ready to gesture to a device as opposed to interacting with our gadgets by touching a piece of glass with our fingers?