In a research presented today at its F8 annual developers conference, Facebook revealed that it took billions of public Instagram photos annotated by users with hashtags, and used that data to train its own image recognition models. According to TechCrunch, the largest of tests used 3.5 billion Instagram images spanning 17,000 hashtags.
The social media giant detailed how it relied on hundreds of GPUs running around the clock to parse the data, and were ultimately left with deep learning models that beat industry benchmarks, the best of which achieved 85.4% accuracy on ImageNet.
Facebook notes that while other image recognition benchmarks may rely on millions of photos that humans have annotated personally, it had to find methods to clean up what users had submitted that could by executed at scale.
The “pre-training” research focused on developing systems for finding relevant hashtags; that meant discovering which hashtags were synonymous while also learning to prioritize more specific hashtags over the more general ones. This ultimately led to what the research group called the “large-scale hashtag prediction model.”
The privacy implications here are interesting. On one hand, Facebook is only using what amounts to public data (no private accounts), but when a user posts an Instagram photo, how aware are they that they’re also contributing to a database that’s training deep learning models for a tech mega-corp?
The sophisticated AI deep learning models this data trained will not only be very useful to Facebook, but could also bring better image search and accessibility tools to users.
