Google Tags Dad as Criminal After Son’s Medical Pics Sent to Doctor

According to a report from The New York Times‘ Kashmir Hill, Google terminated a San Francisco resident’s account after misidentifying photos he took of his toddler’s groin for their pediatrician as child sexual abuse material.

A dad in San Francisco took photos of his toddler’s groin for the doctor. When his Android backed the photos up to the cloud, Google flagged them as child sexual abuse material. He lost his Google account and was investigated by the police. https://t.co/OO4ipXJPsO

— Kashmir Hill (@kashhill) August 21, 2022

Google has measures in place to vet content uploaded to the Google cloud for content pertaining to child abuse or exploitation, much like Apple does for its iCloud service.

Mark, the unfortunate Google customer in question, had his Android smartphone configured to back up all of his photos and videos to Google.

Two days after he took those photos of his son, he received a notification on his phone informing him that his Google account had been disabled because of “harmful content” that was “a severe violation of Google’s policies and might be illegal.”

While digging into the story, Hill came across another case from Texas that was almost identical. In both instances, the fathers lost over a decade’s worth of data they had backed up to their Google accounts.

What’s more, Google flagging their accounts triggered police investigations into both individuals. Each of them was investigated and cleared of any wrongdoing or suspicion.

“I determined that the incident did not meet the elements of a crime and that no crime occurred,” Nicholas Hillard, the police investigator on Mark’s case, wrote in his report.

However, neither of the victims has been able to get their accounts back despite being cleared by the police.

Mark asked Hillard if he could tell Google he was innocent and help get his account back. “You have to talk to Google,” Hillard told him on a call. “There’s nothing I can do.”

The tech giant has no procedures in place to rectify a false positive. “Google stands by the decisions,” Hill said in a tweet.

Google stands by the decisions. Critics said the company needs to allow for corrections. But parents and pediatricians need to be aware of the risks of taking photos like this. “Don’t,” an expert said. If you have to, “avoid uploading to the cloud and delete them immediately.”

— Kashmir Hill (@kashhill) August 21, 2022

The extent of the police investigations, though, is yet another cause for concern.

Mark was informed in December of last year that search warrants had been served on Google and his internet service provider. The police were given access to everything in Mark’s Google account, including his internet searches, his location history, his messages, and any document, photo, or video he had stored with the company.

Experts believe these may not be the two only cases where Google’s algorithms have falsely identified child sexual abuse material. “There could be tens, hundreds, thousands more of these,” said Jon Callas, a technologist at the Electronic Frontier Foundation.