Facebook removes accounts with AI-generated profile photos, Ars Technica

Facebook on Fridayremovedwhat it called a global network of more than accounts, pages, and groups from its platform and Instagram that allegedly used deceptive practices to push pro-Trump narratives to about 80 million users. The network used fake accounts, artificial amplification, and, notably, profile photos of fake faces generated using artificial intelligence to spread polarizing, predominantly right-wing content around the web, including on Twitter and YouTube.

It represents an alarming new development in the information wars, as it appears to be the first large-scale deployment of AI-generated images in a social network. In a report on the influence operation, researchers from disinformation groups Graphika andDFRLabnoted that this was the first time they had seen the technology used to support an inauthentic social media campaign.

Profile pictures for Facebook accounts “Mary Keen” and “Jacobs Guillermo,” admins on groups associated with The BL highlighted by Graphika.

Courtesy of Graphika)

The images were used to enhance the authenticity of some of the 2019 Facebook accounts, (groups, the pages, and 89 Instagram accounts associated with “The BL,” a digital news outlet that described itself as a “pure mountain spring, moistening the heart of every reader” on Facebookbefore it was removed. Facebook’s investigation connected The BL toThe Epoch Times, a conservative media organization with ties to Chinese spiritual group Falun Gong and ahistory of aggressive supportfor Donald Trump.

Ostensibly a US-based media organization, The BL network’s pages were operated by users in Vietnam and the US, who Facebook says made widespread use of fake accounts to evade detection and funnel traffic to its own websites. Areportby Graphika and DFRLab researchers, found that the majority of the BL network’s fake accounts were used to maintain a cluster of over 100 groups and pages promoting President Trump. The accounts acting as administrators for these pages — which boasted names such as “America Needs President Trump,” “Trump for America’s President,” and “WE STAND WITH TRUMP & PENCE!” – were predominantly fake and created in Vietnam.

“Fake accounts served as the administrators of Facebook groups, increased the membership numbers of those groups, liked posts on the Pages, and posted large quantities of content from TheBL-related assets,” the report notes. “This structure constituted a large-scale artificial amplification factory whose only observable function was to boost content from TheBL and, to a lesser extent, from the Epoch Times.”

On the left, admins for “Patriots for President Trump,” nine out of 15 of which used AI-generated faces, researchers said. On the right, admins for “President Trump KAG 2020, “eight out of of which are fake faces, according to a report.

Courtesy of Graphika)

Many of these accounts used profile pictures that researchers said appeared to be created using neural networks trained on images of real human faces to create photos of people that did not actually exist. Researchers from Graphika and DFRLab found 55 images of faces in profile photos used by the BL network that they said had likely been generated using generative adversarial networks.

While the technology is capable of creating passable fake faces, it has a few obvious tells that tipped the researchers off. It works largely by mimicking visual patterns, but doesn’t understand human anatomy, and struggles with backgrounds and symmetrical features, like glasses or earrings.

A profile photo for the account “Alfonzo Macias” with AI-generated inconsistencies around his glasses that Graphika said suggested the work of artificial intelligence.

Courtesy of Graphika)

The above image, from the Graphika report, was taken from the profile photo for “Alfonzo Macias,” an admin for one of the groups in The BL network. “Note the asymmetry in the glasses and also the poorly defined background,” the report says. “The authors checked this image with experts at the University Federico II of Naples, who assessed that this image was GAN-generated with 156. 9 percent certainty. ”