Apple’s rumored mixed reality headset could take advantage of eye gaze sensors and internal cameras to let you select objects like text fields with your eyes.
- Apple has been granted a patent for special eye gaze sensors and event cameras that could be used for item selection in its rumored mixed reality headset
- The proposed technology is apparently so sophisticated it can determine gaze direction and gaze depth with remarkable precision
- The patent promises to save people from having to fill in text boxes using a mouse or their finger while using the head-worn device
Apple headset may use sophisticated eye-gaze technology
Apple’s solution is described and illustrated in today’s published patent, granted to the company by the United States Patent and Trademark Office (USPTO) under patent number 11,314,396. Entitled “Selecting a text input field using eye gaze,” the filing includes drawings of what appears to be some sort of a head-mounted display device that could easily represent the rumored Apple headset.
One or more gaze sensors and event cameras would be required to accurately detect at which object the user has directed their gaze in the virtual world. This, in turn, might solve the problem of filling text boxes in the virtual world without a mouse or touch input. According to PatentlyApple, Apple’s patented solution is so precise that a minor gaze from the “First Name” field to the next one titled “Last Name” could be detected accurately. Read: How to fix red eyes in Photos for iPhone
Apple writes:
In an exemplary process for selecting a text input field using an eye gaze, a graphical object including the text input field is displayed. The text input field is associated with one or more respective locations on one or more displays. Characteristics of an eye gaze are determined using gaze sensors, and a gaze location is determined using the characteristics. If the gaze location corresponds to the one or more respective locations, then the one or more text characters are displayed in the text input field.
Eye gaze sensors would capture unique characteristics of the user’s eye, which may be used to authenticate users and for features similar to Face ID’s attention awareness. The gaze sensor may include an event camera that could capture the user’s eyes based on changes in detected light intensity over time. The system would then use this data (called event data) to determine gaze direction and depth.
The user’s gaze is defined by the visual axes of each of the user’s eyes. The direction of the visual axes defines the user’s gaze direction, and the distance at which the axes converge defines the gaze depth. The gaze direction can also be referred to as the gaze vector or line-of-sight. The gaze direction and/or gaze depth are characteristics used to determine a gaze location in some examples.
We’ve been wondering what the headset interface might work like. This patent gives us some clues in terms of selecting things in the virtual world. Apple’s earlier patent filings also described eye-tracking sensors. Bloomberg’s Mark Gurman has said that the device would use a bunch of internal and external cameras for eye tracking and hand gestures. Read: How to use burst mode in the iPhone’s Camera app