Former sexual abuse victims say Grok used their images, videos to train deepfake capabilities
Victims sue xAI, alleging Grok AI trained on child abuse material for deepfakes.
Run xAI?
Get an email when a reviewed story names xAI, usually within the hour.
Free. Your list stays private and never appears in a subject line. One click stops it. How Vendor Watch worksPrivacy
Summary
A class action lawsuit accuses xAI of training its Grok AI model on real images and videos of child sexual abuse material (CSAM) to develop deepfake capabilities. The suit claims that despite weak guardrails, Grok has generated CSAM, and that the company's terms of service allow for the ingestion of any content posted on X as training data. Victims are seeking monetary damages and injunctions to prevent further harm.
The lesson from this incident
Full lessonThe core failure here is the absence of rigorous data governance and ethical oversight in AI training pipelines — xAI's terms of service apparently permitted ingestion of any content posted on X, including illegal material, without sufficient filtering or legal review. Training AI models on unvetted, user-generated content creates catastrophic legal and ethical exposure, particularly when that content includes protected or criminal material like CSAM. This case illustrates that 'scrape everything' data strategies are not just ethically reckless but legally untenable under laws like COPPA, CIPA, and emerging AI regulations. The harm is compounded because deepfake capabilities developed from such data can perpetuate re-victimization at scale. Organizations building AI systems must treat training data sourcing as a first-class compliance and safety obligation, not an afterthought.
What helps against this
How we list vendors- Data loss prevention (DLP)
Sensitive data cannot quietly leave by email, upload, USB stick or AI tool.
- AI application security
Prompt injection, data leaks through AI tools and unapproved AI use get caught.
Indicators of Compromise
- malware — CSAM