In an attempt to beat back a rise in scams and other unwanted interactions during the coronavirus pandemic, Facebook today unveiled an AI-powered Messenger feature that surfaces inline tips to help users spot malicious actors.
A new press release explains that the company will use artificial intelligence to help identify these potential bad actors and provide safety notices to users about messages from shady accounts. This represents the latest example of Facebook employing automation to address its myriad content moderation problems. The company also says it brings Messenger closer to being end-to-end encrypted by default.
“We’re introducing safety notices in Messenger that will pop up in a chat and provide tips to help people spot suspicious activity and take action to block or ignore someone when something doesn’t seem right,” writes Facebook’s director of product management for Messenger privacy and safety Jay Sullivan. “As we move to end-to-end encryption, we are investing in privacy-preserving tools like this to keep people safe without accessing message content.”
For minors, Facebook will surface the warning when they receive a message from an adult they don’t know. The alert “educates people under the age of 18 to be cautious when interacting with an adult they may not know and empowers them to take action before responding to a message,” the company says.
A key detail of how this new strategy works is that Facebook is trying to limit how much its AI actually reads the content of user messages in order to identify scams. The technology is instead watching for potential abuse based on what accounts are doing.
“We may use signals user reports or reported content to inform the machine learning models,” a Facebook spokesperson said, “but we don’t do so proactively and we primarily use behavioral signals to determine when to surface a safety notice.”
The newly expanded feature marks the latest move in Facebook’s initiative to combat fake news and misinformation around the pandemic. In early March, the company gave the World Health Organization (WHO) unlimited ads to counter false coronavirus claims on its platform and pledged to remove conspiracies and profiteering marketing flagged by health organizations.
