Apple Granted Patent for 3D Hand-Tracking TrueDepth Technology

Apple has been granted a patent for technology that shows the Cupertino company is exploring a Mac/iPhone feature that can identify hand movements and gestures.

According to a new report from Patently Apple, the patent, which Apple initially applied for in August 2016, is for technology developed by Israeli startup PrimeSense, a company that Apple acquired back in 2013. PrimeSense also developed Microsoft’s Kinect sensor.

The patent shows that Apple is exploring possibilities in using the iPhone X‘s TrueDepth camera system – which is capable of recognizing a user’s face by projecting 30,000 infrared dots – to effectively recognize user hand movements.

One can imagine that Apple would likely use the technology to track hand movements and gestures in a three-dimensional space. The patent describes a number of example gestures one could use to control the interface.

As seen in the image above, a few of the gestures are called “push,” (figure 2) “wave,” (figure 3) and “up” (figure 5) interactions. According to the patent, these sort of interactions could be used for basic functions within the iOS interface:

Gestures described herein include focus gestures and unlock gestures. A focus gesture enables the user to engage (i.e., take control of) an inactive non-tactile 3D user interface. An unlock gesture enables the user to engage a locked non-tactile 3D user interface, as pressing a specific sequence of keys unlocks a locked cellular phone. In some embodiments, the non-tactile 3D user interface conveys visual feedback to the user performing the focus and the unlock gestures. 

Examples of unlock gestures include an “up” gesture (e.g., raising hand 30 a specified distance), a sequence of two sequential wave gestures, and a sequence of two sequential push gestures, as described in detail herein below.

The patent continues to emphasize the potential importance of these sorts of applications, saying that “The ability to understand human gestures, and, in particular, hand gestures, as they relate to [human computer interaction], is a very important aspect in understanding the intentions and desires of the user in a wide variety of applications. In this invention, a novel system and method for three-dimensional hand tracking is described.”

While one at this point cannot be certain what sort of applications Apple would use this technology for, it’s exciting to see what kind of plans Apple has in store for us in the near future.