Facebook has announced the introduction of automatic alternative text, a feature that aims to make the billions of photos uploaded to Facebook, Instagram, Messenger, and WhatsApp accessible to people who are blind or severely visually impaired.
Before automatic alternative text was added to Facebook, blind people would only hear the name of the person who had shared a photo, followed by the term “photo” when they navigated the News Feed.
Now they should feel less excluded from the visual content shared by users: Automatic alternative text generates a description of a photo using advancements in object recognition technology. The feature is available on iOS devices today and will hit Android later.
As detailed by The Verge, automatic alt text recognized objects in photos using machine learning, which helps build artificial intelligences by using algorithms to make predictions. For example, showing a piece of software enough images of a dog will enable it to identify a dog in a picture – with time.
The new feature identifies things in Facebook photos and uses the iPhone’s VoiceOver feature to read out descriptions of the image to users.
The technology was developed by Facebook’s 5-year-old accessibility team led by Jeff Wieland. Verge reporter Casey Newton traveled to Facebook’s accessibility lab to see the feature in action and gathered other tidbits as well:
In a demonstration, King pulled up a few stories on Facebook that include photos. He set the screen to black so we couldn’t see anything. If you’d like to re-think everything you ever thought you knew about web design, watch a blind person use the internet for five minutes. King normally has his screen reader speak to him incredibly quickly — the slightest audio cues now orient him on the page, reading Facebook posts out loud, identifying links, and exposing various buttons. His fingers were a blur as he entered commands on a standard MacBook Air. I remained totally lost until King turned the screen back on, save for the handful of words that described what we were seeing on Facebook. One Facebook post had a photo with the caption “Sunday night splurge,” and the description read aloud by the phone was “pizza, food.” When King turned the screen back on, there was a photo of a giant pepperoni pizza with olives
What’s interesting is that using this technology Facebook will suggest only a tag for a photo if it is 80% sure that it knows what it is looking at.
The automatic alternative text is currently available only on iOS and only in English, but the team is already pushing for the next step: recognizing objects in videos and something they call “visual Q&A”, which allows users to ask questions about images and receive answers from Facebook’s artificial intelligence.
