A Major Gaze-Tracking Patent published last week was written by one of Apple’s Mixed Reality, Computer Vision Superstars


Protect Your Access to the Internet


Last Thursday the US Patent & Trademark Office published a patent application from Apple that relates to assessing characteristics of pupils of eyes, and in particular, to systems, methods, and devices for assessing pupil characteristics using light reflected off the eyes to estimate the user’s gaze direction using multiple glints to identify the user’s eye shape, position, and orientation. In various implementations, pupil characteristic assessment is used to facilitate gaze tracking, which may be used to enable user interaction.


Last week Apple analyst Ming-Chi Kuo stated that “Apple’s future headset will include innovative human-machine UI technologies such as gesture control, object detection, eye/gaze tracking, iris recognition, voice control, skin detection, expression detection, and spatial detection.”


Patently Apple has covered a number of eye and gaze tracking patents (01, 02, 03, 04 & 05) and it’s clear that this technology will play an important role in Apple’s future mixed reality headsets. Apple fans will appreciate this technology once they’re able to see it in action. For now, Apple’s patent on this technology is very technical and likely to be more appreciated by developers and engineers in this field.


Apple notes in their patent filing that existing pupil detection techniques may use an image sensor and integrate light intensity level over an exposure period to produce greyscale images and then attempt to detect the pupil using the greyscale images. The pupil is detected based on greyscale contrast between the pupil region and the surrounding iris region and thus rely on there being significant contrast between the pupil and the iris. Such pupil detection techniques may not be as accurate or efficient especially in circumstances in which the contrast between pupil and iris is less significant. This is what Apple’s invention sets out to remedy.


Pupil Assessment using Modulated On-Axis Illumination


Apple invention covers a pupil characteristic assessing system that includes a light source, an image sensor, and a processor that performs pupil characteristic assessments on data received from the light sensor regarding light from the light…