Microsoft’s ‘Seeing AI’ Talking Camera App Hits the App Store

Designed for the blind and low vision community, ‘Seeing AI’ talking camera app has just been released in the App Store in select countries, including the U.S., Canada, India, Hong Kong, New Zealand and Singapore. 

“Seeing AI is a free app that narrates the world around you”, says Microsoft, adding that it’s an ongoing research project that harnesses the power of AI to open up the visual world and describe nearby people, text, and objects.

“I really do think this app will be a game changer for many as it’s just simple to use,” said Jenine Stanley, who is legally blind and reviews apps and technology for Main Menu, the American Council of the Blind’s tech radio show.

Stanley said she was “blown away” by the speed and ease of use of the app. One of her favorite features is the Short Text channel, which reads signs, labels and other short snippets of text. “That feature alone is worth the app,” she said. “It’s also fast — almost instant.”

The app uses artificial intelligence and the camera on your iPhone to perform a number of useful functions:

• Reading documents, including spoken hints to capture all corners of a document so that you capture the full page. It then recognizes the structure of the document, such as headings, paragraphs and lists, allowing you to rapidly skip through the document using voiceover.
• Identifying a product based on its barcode. Move the phone’s camera over the product; beeps indicate how close the barcode is – the faster the beeps, the closer you are – until the full barcode is detected. It then snaps a photo and reads the name of the product.
• Recognizing people based on their face, and providing a description of their visual appearance, such as their gender, facial expression and other identifying characteristics.
• Recognizing images within other apps – just tap Share, and Recognize with Seeing AI.

Check out the following introductory video for Microsoft’s new ‘Seeing AI’ app for iOS: