ARKit

This awesome ARKit app uses HTC Vive to bring mixed reality to iPad

Unlike virtual reality which completely immerses you in computer-generated images, augmented reality superimposes virtual objects on top of your world rather than close it out.

Mixed reality, on the other hand, (sometimes called hybrid reality) merges the two worlds so that physical and digital objects co-exist and interact in real time.

A new recently published demo combines an ARKit-driven app running on the Unity 3D engine on an iPad Pro with real-time input from HTC's Vive virtual reality headset and controller. Created by NY-based virtual reality developer Normal VR, the app features a cute avatar drawing a virtual painting in the middle of the company's office.

What's really interesting about it is the fact that a person wearing a HTC Vive is revealed when the camera pans right, proving that the app captures the moves of the person in real-time and projects the resulting actions in their real world through augmented reality.

“This is going from East Coast to West Coast (server) and back. We definitely do some extrapolation to account for ping,” developers wrote. You could easily imagine the possibilities for remote interactions between people across the globe in mixed reality.

Blobbing in the studio today w/ the Vive + ARKit. Definitely some huge mixed reality potential here. #arkit #vr #indiedev #gamedev pic.twitter.com/C1zANBuSrx

— Normal (@normalvr) July 10, 2017

Not sure about you, but I'm really impressed by this demo.

There's a lot going on here as this app brings objects from the virtual world into the real one, in real-time—not only does the avatar realistically replicates the person's actions, it shows the digital painting on top of the real world as it's being created, updating it constantly.

Welcome to the future of mixed reality!

We showed you many demos powered by Apple's ARKit framework for building augmented reality apps on iPhone and iPad, ranging from a simple but awesome measuring tape to home decor shopping, accurate room measurement, Tic Tac Toe, food ordering and more.

Even Ikea has jumped on the AR bandwagon by partnering with Apple on an ARKit-powered app that will let you try out virtual furniture in your home, with support for in-app ordering.

The beauty of ARKit is that it does all the heavy lifting, allowing developers to focus on app design rather than deal with things like plane detection, lighting estimation, tracking, etc.

Using computer vision and other techniques, ARKit does all that by analyzing live camera feed and data from iPhone or iPad's built-in sensors. ARKit requires the A9 or A10 chip, meaning augmented reality apps will require an iPhone 6s or newer or one of the latest iPad models.

Augmented reality is the future of home decor shopping

A great little ARKit-powered hackathon project by Melbourne, Australia developer Jacqui Hyslop of RedBubble shows off the tremendous potential of Apple's new framework for building augmented reality experiences for iPhone and iPad with iOS 11.

The simple proof-of-concept app allows the user to place virtual cushions (and other home decor items) on a couch and other pieces of furniture in their real world, but without the psychedelic surface-shifting effect plaguing many other AR platforms.

This app clearly proves that home decor shopping in augmented reality beats having to drive to a store just to see how that cushion would look like on a bed that isn't yours in a showroom that shares little resemblance to your own home.

The video demonstration, embedded below for your viewing pleasure, highlights the full sophistication of ARKit's reliable plane and object detection and precision tracking. It's remarkable that all of this is happening so fluidly without any special hardware involved.

https://www.youtube.com/watch?v=bwNAykf4IKw

ARKit uses machine learning, artificial intelligence, computer vision and sophisticated algorithms to analyze live camera feed and “see” the world around you. It also taps into data from on-board device sensors, such as the accelerometer and gyroscope, to detect horizontal planes in your real world, determine local lighting conditions and so forth.

Those kinds of ARKit-powered apps should be even better on iPhone 8 due to the rumored rear-facing 3D laser sensor that will allow the device to accurately sense depth and map objects.

Be sure to watch another sophisticated ARKit demo showing off an app that basically allows the user to map out the floorspace by walking around a room. It calculates total footage of the room and may even produce detailed blueprints that could be exported to CAD apps.

And here it is.

https://www.youtube.com/watch?v=iw9MPZoPqCQ

The future of furniture shopping looks bright, too, as furniture giant Ikea has partnered with Apple on an ARKit-powered app of its own that promises to let users try out virtual furniture at home before placing an actual order right inside the app.

Just imagine the possibilities: with Ikea's app, you could make sure that the bookshelf you've been eyeing for some time would fit in that corner—again, without being bothered to actually, you know, visit one of their stores.

Are you excited for ARKit-powered apps?

ARKit demo: accurate room measurement in augmented reality

If the augmented reality (AR) tape measure app we recently featured has gotten you excited for the possibilities that Apple's ARKit framework brings to developers, you're wholeheartedly recommended to watch this video demonstration of an upcoming ARKit-driven app that will let you take precise room measurement by pointing your iPhone's camera at the corners.

The app will help generate 3D models and automatically calculate the footage for the user. It's a really cool-looking demo and I can't wait to try this app out when it releases.

Watch the demonstration video below.

https://www.youtube.com/watch?v=iw9MPZoPqCQ

The app was created by a company called Smart Pictures, which provides the Measurement Cloud system that enables dimensioning intelligence for the interior world.

Here's a recent ARKit-powered tape measure demo that set the Internet on fire.

https://www.youtube.com/watch?v=z7DYC_zbZCM

This level of precision is made possible by ARKit's accurate plane detection and reliable object tracking. On existing devices, the ARKit framework leverages data from on-board sensors and uses computer vision algorithms constantly analyzing live camera feed.

ARKit-driven apps should be even better on iPhone 8 because Apple's OLED iPhone is said to feature a dedicated 3D laser sensor on both its front and back for even better augmented reality features and faster, more precise autofocus.

ARKit demos: jumping between planes, people tracking, alien invasion, Tic-Tac-Toe & more

Apple's ARKit framework is slowly but surely emerging as one of the best new features in iOS 11. Many developers have built everything from virtual tape measures and Minecraft to ballerinas made out of wood dancing on floors. It's remarkable that most of the ARKit demos we've seen so far were built in a matter of hours or days, not weeks or months.

Today, we want to highlight a few additional ARKit demos that we've curated. These videos highlight ARKit's incredibly reliable and accurate tracking features that don't require any special hardware beyond the sensors and the camera already present in your iPhone or iPad.

First up, we have this demo showing jumping between different planes.

https://www.youtube.com/watch?v=pj3PbRTgpQk

ARKit automatically detects horizontal surfaces, called planes, such as tables and floors, and can track and place objects on smaller feature points as well. This is all handled automatically, with uncanny precision, using only data from your iOS device's camera and sensors.

The following pair of videos demonstrate a virtual character interacting with the environment by autonomously jumping a flight of stairs and between different surfaces of the real world.

https://www.youtube.com/watch?v=vozrtqe7MZY

https://www.youtube.com/watch?v=o7REJj_bN-c

Interactions between virtual objects and real people are easy as a pie with ARKit.

https://www.youtube.com/watch?v=RdAqD4uIbX0

The Tracking Monster demo, seen below, uses ARKit and the Unity engine to track a monster with the dynamically updated shadows based on changing lighting conditions in the real world.

https://www.youtube.com/watch?v=_IYT-OCWqDg

Maze games will never be the same!

Speaking of games, here's Tic-Tac-Tio in augmented reality, developed By Bjarne Lundgren.

https://www.youtube.com/watch?v=IBBq473vuMo

And last but not least, Mixed Reality Design posted the following example of an augmented reality app depicting an alien vessel hovering ominously above a construction site.

Hollywood will soon invade your smartglasses airspace → fact https://t.co/jLYm1YcBW2 pic.twitter.com/to2qqfFIVr

— Mixed Reality Design (@MixedrealityD) July 4, 2017

Head over to the Mixed Reality Design's Twitter account for more AR examples like this.

While it's not entirely clear that this particular demo uses ARKit, it does highlight the possibilities for AR movie trailers that could be coming soon to your phone.

Be sure to check out other interesting ARKit apps and demos, including an upcoming furniture-ordering app from Ikea, a measuring tape that blew up on the web, an ARKit-powered VR mode in Maps, an inter-dimensional portal and much more.

“I think there is a gigantic runway that we have here with the iPhone and the iPad,” Apple executive Greg Joswiak said of ARKit in a recent interview with The Australian. “The fact we have a billion of these devices out there is quite an opportunity for developers.”

How do you like these ARKit demos? Which one is your favorite, and why? Chime in with your thoughts and observations in the comments section below.

New ARKit demos: Tesla Model 3, food ordering & inter-dimensional portal

With the new ARKit framework for building augmented reality (AR) apps, Apple is turning existing iOS devices into the largest AR platform in the world.

We previously shared a bunch of interesting ARKit demos showcasing the possibilities of the framework, including an upcoming furniture-ordering app from Ikea, a measuring tape, an ARKit-powered VR mode in Maps and more.

Today, we're highlight three more examples of ARKit-driven apps: a Tesla Model 3 app by an impatient fan, an example of an inter-dimensional portal in the middle of the street and an app that promises to change how we order food.

First up, a demo app by an impatient Tesla fan who couldn't wait for his ordered Model 3 so he made this ARKit-powered app that lets him drive around a virtual Model 3 in his real world, activate the headlights and so forth.

@elonmusk Couldn't wait 4 my #Model3, so made this AR app, what do you think? #ARkit pic.twitter.com/lIRLTZox7N

— Jelmer Verhoog (@JelmerVerhoog) July 1, 2017

Food ordering will never be the same, says developer Alper Guler who created an ARKit-driven app which renders various foods on your table that you can pan around, zoom in and out, rotate and more.

https://www.youtube.com/watch?v=oFdgVNg4ryM

And last but not least, we have French consulting agency Nedd which came up with a great example of AR+VR, via iPhoneAddict.fr.

https://www.youtube.com/watch?v=rIPfpGCxONQ

Before signing off, here's a quick volumetric capture example with ARKit.

hint pic.twitter.com/WOuqVer1Ph

— Made With ARKit (@madewithARKit) June 27, 2017

Very impressive so far, don't you think?

New demos show how easy it is to bring 3D models to life with ARKit

One of the best aspects of ARKit, Apple's new framework for building augmented reality apps, is the fact that it does all the incredibly complex heavy lifting like detecting room dimensions, horizontal planes and light sources, freeing up developers to focus on other things.

ARKit analyzes the scene presented by the camera view, in real time.

Combined with sensor data, it is able to detect horizontal planes, such as tables and floors, as well as track and place objects on smaller feature points with great precision.

And because it uses the camera sensor, ARKit can accurately estimate the total amount of light available in a scene to apply the correct amount of lighting to virtual objects.

First, check out this demo from Tomás Garcia.

https://www.youtube.com/watch?v=1G2YbQuQHps

Children's bedtime stories will never be the same come this fall!

Another developer has put together a quick demo showing off his AI bot, named “Pepper”.

https://www.youtube.com/watch?v=brFKo_tSkw8

According to the video's description:

I've been working on an AI bot for a while now. To be short, it's like V.I.K.I in the movie “I, Robot”. With the help of ARKit, I was able to bring it close to a real life assistant.

Due to obvious reasons, I'm not demonstrating her functionality in this video. So I ended up showing you guys how easy and simple it is to bring 3D models to life with Apple's new framework.

These videos clearly demonstrate how easy ARKit makes it for developers to match the shadows of their virtual objects to lighting conditions in the real world.

https://www.youtube.com/watch?v=lQt96saECfM

Here are some additional ARKit-enabled demos.

https://www.youtube.com/watch?v=-o7qr1NpeNI

 

https://www.youtube.com/watch?v=sJR_f7XCuvA

 

https://www.youtube.com/watch?v=lQt96saECfM

ARKit requires A9 or A10 processors, meaning ARKit apps will require an iPhone 6s or newer or one of the latest iPad Pro models, either the 9.7-inch or the 10.5-inch one.

If anything, these videos demonstrate just how easy Apple has made it to put together an AR app. Are you looking forward to ARKit, and why? Leave a comment below to let us know.

Watch new ARKit demos: Minecraft and measuring tape

The website madewitharkit.com dedicated to highlighting cool apps made with Apple's new ARKit framework, was updated today with a pair of new video demonstrations showing off some of the augmented reality possibilities coming to iPhone and iPad with iOS 11 this fall.

The first demo has the user selecting two spots in the real world, as viewed through an iPhone's lens, to calculate the distance between them, transforming the device into a working tape measure. That's a great example of the power of the ARKit framework.

https://www.youtube.com/watch?v=z7DYC_zbZCM

The app was built by Laan Labs and, like other ARKit-enabled apps, uses an iOS device's camera along with sensor data to precisely find horizontal planes in the real world, such as tables, floors and other objects.

You can beta-test the app by signing up at armeasure.com.

Measure distances with your iPhone. Just because you can. Clever little #ARKit app by @BalestraPatrick https://t.co/b2mXe2FS84 pic.twitter.com/pyoHp99Yts

— Made With ARKit (@madewithARKit) June 25, 2017

Laan Labs has other examples of proof-of-concept apps built using ARKit on their Twitter, like the following example of impressive 3D drawing in augmented reality.

https://twitter.com/laanlabs/status/878692051889655808

As for an AR-enabled Minecraft, we don't know if Minecraft creator Mojang is working on one, but that didn't stop developer Matthew Hallberg from recreating Minecraft in AR using the ARKit framework and the Unity engine.

By superimposing Minecraft building blocks on top of the real-world, and taking advantage of ARKit's super accurate tracking, the user is able to walk around their environment and place Minecraft blocks at arbitrary spots. “I love that you are able to place life size objects because the tracking with ARkit is so good,” Matthew said.

https://www.youtube.com/watch?v=qFGx9QcE5Gk

Apple is also using ARKit tracking for an impressive virtual reality mode in Apple Maps on iOS 11. The Cupertino giant is even helping Ikea build an ARKit-powered app which will let you try out virtual furniture at home before purchasing it.

ARKit requires a device with an Apple A9 or A10 chip because those processors deliver “breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes,” as per Apple.

How do you like the aforementioned ARKit demos? Are you looking forward to augmented reality-enabled apps, and why? Chime in with your thoughts in the comments section.

iOS 11 Maps has crazy cool VR mode that lets you move around by walking

Apple Maps on iOS 11 beta 2 features a great new virtual reality (VR) mode that takes advantage of Apple's new ARKit framework to let you move around in 3D by walking.

This unapologetically cool feature seems to be tied to Flyover, which replaces satellite imagery with three-dimensional buildings, landmarks and other points of interest.

The new VR mode on iOS 11 Maps was highlighted yesterday by Twitter user @StijnDV, but it appears to have been originally discovered by Tweetbot developer Paul Haddad on Wednesday.

To try it out yourself, open Maps on iOS 11 beta 2, switch to 3D mode by tapping “3D”, then use the search field at the bottom to find a place that has Flyover.

On the place card, tap the Flyover button and move the device around to rotate the view. Better still, why don't you actually move forward, backward or side to side to explore the map in VR?

Mind blown.

So, how do we know this nifty feature actually uses ARKit? Because it displays a message when you cover the camera, just like any ARKit-powered app does, saying you should aim the device at a different surface because “more contrast is required”.

As a quick backgrounder, ARKit analyzes live camera feed in real-time, using computer vision to find horizontal planes in your real world, such as tables and floors. I was able to successfully test the feature on my iPhone 6s running a second beta of iOS 11. Because I don't currently own an iPad, I couldn't test VR mode in Maps on the Apple tablet.

WOW There is an VR mode in Apple maps on iOS 11! It seems to use ARKit for positioning! pic.twitter.com/IdXiGoed26

— Stijn (@StijnDV) June 24, 2017

At any rate, this appears to be the default mode for Flyover now, not a special setting. But don't you worry, there's the option to switch back to the old Flyover mode where you rotate and zoom your Flyover view using touch interactions.

This is honestly one of the coolest features in iOS 11! pic.twitter.com/Zjr6RRkKHk

— Stijn (@StijnDV) June 24, 2017

This is a wicked cool feature and I cannot help but wonder how it might look like when experienced through Apple's rumored digital glasses that, as per Robert Scoble, should use optics by German lens specialist and optical instruments maker Carl Zeiss.

You can actually move around by walking! This is crazy cool! pic.twitter.com/ttR6RaAo7D

— Stijn (@StijnDV) June 24, 2017

Some people couldn't get Maps' new VR mode to work, but I suspect it may have something to do with their hardware. Maps' VR mode uses ARKit, which tracks your actual position in the real world with the camera but requires newer hardware.

Holy Flyover Magic Window batman. pic.twitter.com/Fb8nPeLT5J

— Paul Haddad (@tapbot_paul) June 22, 2017

According to Apple, ARKit runs on the Apple A9 and A10 processors. “These processors deliver breakthrough performance that enables fast scene understanding and lets you build detailed and compelling virtual content on top of real-world scenes,” says the company.

In other words, anything older than iPhone 6s, iPhone 6s Plus, iPhone 7, iPhone 7 Plus, the 9.7-inch iPad (early-2017 model) or iPad Pro won't be able to run iOS 11 Maps' VR mode.

So, is this cool or what?

We'd obviously love to hear your thoughts and predictions regarding iOS 11 Maps' new VR mode and what it might signify in terms of possible new VR hardware from Apple.

Do us a favor and chime in with your thoughts in the comments section.