Apple has been granted a patent for immersive video streaming for a future mixed-reality headset, which is supposed to be one of its killer features.
- What’s happening? Apple has won a patent that relates to immersive video content for a “virtual reality headset or an augmented reality headset.”
- Why care? The patent backs a recent report from Bloomberg’s Mark Gurman about immersive video watching being one of an Apple headset’s killer apps.
- What to do? Enter “11570417” on the USPTO patent search website if you want to explore the granted patent in more detail.
Patented: Immersive video streaming for an Apple headset
PatentlyApple reports that the United States Patent and Trademark Office (USPTO) has granted Apple a patent for immersive video streaming on a head-worn device.
Titled “Immersive video streaming using view-adaptive prefetching and buffer control,” it notes in the description that immersive video content can be presented in three dimensions using a wearable display device. Apple describes a head-worn display device as “a virtual reality headset or an augmented reality headset.”
An unreleased Apple headset would employ its many sensors to accurately determine the position and orientation of a wearer’s body to figure out which portions of the immersive video content to present at any given moment.
Apple offers examples of the wearable display devices that could be used:
- A wearable headset, such as a virtual reality headset
- An augmented reality headset
- A mixed-reality headset
- A wearable holographic display
Apple is thought to be working on a mixed-reality device said to support both virtual reality, which completely engulfs the viewer in virtual worlds, and augmented reality, which layers computer-generated information and graphics on top of the user’s real-world view. Read: How to force iPhone to stay in low-power mode
Types of head motions
“A viewport can be selected to present a portion of the immersive video content to the user (e.g., based on the position and/or orientation of the user’s head) to give the user the impression that they’re viewing the visual data according to a particular field of view and/or viewing perspective,” Apple explains.
This viewport can be dynamically updated as a headset wearer moves their body and tilts their head around to give the user “the impression that they’re shifting their gaze within a visual environment.”
A headset would need to constantly analyze data from onboard sensors to determine the position and orientation of your head in multiple dimensions. The sensors, Apple explains, should be able to detect the following head movement:
- Surge motion: A user translates their head in a forward or backwards direction (x-axis)
- Sway motion: A user moves their head in a left or right direction (y-axis)
- Heave motion: A user translates their head in an upward or downward direction (z-axis)
- Roll motion: A user rotates their head about the x-axis
- Pitch motion: A user rotates their head about the y-axis
- Yaw motion: A user rotates her head about the z-axis
How immersive video on an Apple headset might work
In a recent report on Bloomberg titled “How Apple’s Upcoming Mixed-Reality Headset Will Work,” reporter Mark Gurman describes how enjoying immersive video with an Apple headset strapped on your face might work.
Apple is planning for the headset to have a dedicated video-watching feature that can make viewers feel like they’re seeing a movie on a giant screen in another environment, such as a desert or outer space.
The company is apparently also working on updating its own Apple TV+ content to work with a yet-to-be-announced device. Apple also bought streaming company NextVR in 2020 to create sports content in virtual reality.
Apple Reality Pro and Reality One
German’s Apple headset reporting claims that the first version of a future device could be marketed as “Reality Pro.” According to him, the inaugural product will be packed to the gills with advanced technology like sensors for eye, head and hand tracking, two micro-OLED displays with 8K resolution, a custom chip and more.
It’s said to resemble ski goggles, cost around $3,000 and be announced in the spring—with developers being the primary target market. A less expensive version could arrive later, priced around $15,000 and dubbed “Reality One,” once developers have created enough apps to enjoy in virtual or augmented reality.