Meta Failed to Catch Hundreds of AI Child Abuse Ads. Some Included Images of Real Kids

Summary

Meta has been criticized for failing to detect and remove hundreds of ads containing child sexual abuse material (CSAM) across its platforms, including Facebook, Instagram, and Threads. Researchers discovered that some of these ads, which were often AI-generated or manipulated images of real children, remained live for extended periods and linked to harmful 'nudification' apps. This failure occurred despite Meta's stated efforts to improve detection with new AI tools, prompting planned investigations by lawmakers.

Full text

CommentLoaderSave StorySave this storyCommentLoaderSave StorySave this storyEditor’s note: This article contains descriptions of imagery depicting child sexual abuse. Reader discretion is strongly advised.Early last month, Meta deleted around 50 ads posted on Facebook, Instagram, and Threads that directly included child sexual abuse material (CSAM), following reporting by WIRED. At the time, the company said that most of the abusive ads had been published prior to it rolling out new AI tools designed to “better detect and block” harmful ads being uploaded. That AI system, however, doesn’t appear to have been entirely effective.Instead of spotting a decline in the abusive ads in recent weeks, researchers say they have now discovered hundreds more of the disturbing ads containing graphic sexual scenes involving children—including some that were identical to those Meta recently removed. More than 250 additional ads containing CSAM have run across Meta’s social media platforms since the start of August, according to researchers at the nonprofit Tech Transparency Project who shared its findings with WIRED. Unlike the 53 ads TTP researchers initially discovered, they were able to identify images of real children in the new ads.These included sexual content depicting a minor belonging to a European royal family, transforming an officially released photograph into a video of a “graphic sex act.” The researchers are not naming the royal to protect the victim and say they contacted officials in the country. WIRED confirmed the source image came from the official website of a royal family.“What we’re talking about here is not simply AI-generated child sex abuse material,” says Katie Paul, the director of the TTP. The researcher says it appears some images have been taken from the web before being used in the ads. “There’s a lot of harm that can come from this all while Meta is raking in money in the process.”Overall, the researchers discovered that Meta has published more than 350 of the abusive video ads since the end of last year, with many of them linking to harmful harmful “nudification” apps linked to Chinese developers. In each instance, the researchers immediately reported the ads to US child safety authorities and through Meta’s reporting tools when ads were live and being shown to users at the time of their discovery. The researchers shared detailed findings with WIRED, which independently identified hundreds of other related “nudification” ads containing explicit content of adults on Meta’s platforms.While the child abuse ads have now been removed for violating Meta’s policies on child exploitation, adult exploitation, and nudity, all of the ads were originally published despite the company’s policies saying it reviews all ads that run on its platforms. In some instances, the TTP researchers say, it took Meta a week to review live ads they had reported for containing CSAM with Meta’s tools. During that time, the ads racked up hundreds more views, according to screenshots seen by WIRED.When some ads containing child sexual abuse were removed, TTP says, they did not initially include disclosures indicating that they violated Meta’s child sexual abuse policies; those details only appeared to be added after WIRED asked Meta about the discrepancy.The cache of video ads published across Facebook, Instagram, Messenger, Threads, and Meta’s broader advertising network since the end of last year follow similar patterns. Many include an innocuous picture of a preteen or teenage girl—only one contained a boy, according to the researchers—and overlaid text saying it is “not just a photo” and “there are no restriction” [sic] on what can be done with the image. When the seconds-long video ads start playing, the children’s faces are morphed into graphic sexual videos, often showing them performing sex acts. When clicked, the ads directed people to download AI-face swapping or video apps from Apple and Google’s app stores. “This is the AI that men actually use,” many of the ads say.Paul says the researchers were able to find the real-world identities of four minors whose images were used in the ads, three of which are from the US. “There were several photos from the same teen who has a social media influencer account on one of Meta’s platforms,” Paul says. The second instance involved another teen influencer who had a public account, the researcher says. “One was a stock photo model that was a real person,” Paul says of the third instance. “It was actually labeled as pre-teen on the stock photo websites.” The final instance was the European royal family member.The researchers reported all of the ads they found to the National Center for Missing and Exploited Children’s (NCMEC) CyberTipline, which allows people to report online child sexual abuse material, and say they are consulting on how to appropriately disclose their discoveries to the those impacted.The ads were collectively shown to more than 29,000 accounts in countries across the European Union, plus around 7,000 accounts in the United Kingdom, according to data in Meta’s ad library, which shows live and archived ads, including the harmful ads that were no longer being shown to people. The ads were likely seen by thousands more people than Meta’s ad library indicates, with more than 250 of them appearing in the United States, 120 in Australia, and 80 in India, where Meta does not publish granular ad performance data.“We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated,” says Meta spokesperson Tracy Clayton. The company has previously removed hundreds of thousands of ads for violating its policies on nudification. The statement claims that “many” of the ads found by the researchers had already been removed and most had fewer than 200 impressions, with the company receiving “under $5,000” in ad payments.In an email to WIRED, Meta says it reports child exploitation content to NCMEC and is reporting the ads to the organization. Meta minimized the findings by saying that the AI-generated videos featured adults and the clips of the minors were brief or blurred, which made detection of the content more difficult. “The criminals behind this activity know we’ve built strong defenses, including systems that block violating ads as well as ongoing monitoring in case we initially miss something,” Clayton says.The TTP researchers’ discovery of hundreds of ads containing child abuse material comes amid Meta paying up to $16.7 billion to settle lawsuits over its social networks allegedly harming children. Meta is also appealing a New Mexico court decision that, if enacted, would require it to improve its CSAM reporting processes and include human reviews in the state.Multiple lawmakers and regulators now say they are investigating or looking into the ads following WIRED’s reporting. “It is disappointing that Meta continues to run ads featuring child sexual abuse material and nonconsensual intimate images after it was brought to their attention,” says Virginia senator Mark Warner, who sent Meta CEO Mark Zuckerberg a letter following WIRED’s initial reporting. The letter has not been replied to yet, according to Warner’s office. “Meta needs to follow their own advertising standards, heed the call, and put an end to these illegal and inappropriate ads,” he says. (Meta tells WIRED it will reply to the letter).A spokesperson for Michigan’s attorney general says their office “is aware of reports concerning AI‑generated child sexual abuse material appearing in advertising on Meta. This is extremely troubling, and we are actively looking into the matter.” Similarly, Florida attorney general James Uthmeier posted on X that his office is “investigating.”Meanwhile, a spokesperson for Australia’s eSafety internet regulator says it is “highly concerned” by the reports and that it is assessing Meta’s compliance with its rules. It says it has requested “information on this matter from the company

Entities