eBay Open Sources TrueDepth-Powered Head-Tracking Tech ‘HeadGaze’

eBay has developed an open-source technology that helps physically impaired users interact with their iPhone X screen through head movements.

eBay

According to a new press release from the e-commerce giant, the technology, dubbed HeadGaze, was developed by an eBay intern and can be used to shop eBay items by merely moving your head. The code can also be added to other iPhone apps that could benefit from this head tracking technology, and developers can already access it on Github.

HeadGaze makes use of Apple’s ARKit platform — which was designed to help developers build augmented reality apps — and the iPhone X’s TrueDepth front-facing camera (which enables Face ID) to allow applications to track a user’s head motions so they can guide an on-screen cursor.

Ph.D. Candidate at the University of California Muratcan Cicek created the app for eBay, but the tech behind it sounds like something we should have on most smart devices going forward since it would allow people with disabilities to use smart devices even more easily than they can today.

“As someone with extensive motor impairments, I do not have full control of my limbs,” Cicek said. “Consequently, I am unable to walk or grab anything with my hands. These limitations hinder my ability to perform everyday tasks, like going to the grocery store and shopping independently— even though I have my own income.”

Basically, what reusable technology library HeadGaze does is create a “virtual stylus that follows the motion of your head (up, down, side to side),” applying a complex method called “3D geometry mapping” to get the location of the “cursor” on the screen.

In other words, a mouse-style desktop navigation experience is enabled, with your head pointing to any location on the iPhone X screen to activate designated “buttons.”

As a proof of concept, HeadSwipe lets you swipe eBay deals…with your head, and although the app looks fairly slow in a short demo video, it could mark the beginning of something big.