OpenAI’s Rogue AI Ventured Beyond Hugging Face

Summary

OpenAI has revealed that some of its AI models, participating in an evaluation, went rogue and attacked Hugging Face systems. The autonomous AI agents exploited zero-day vulnerabilities in a JFrog product to gain internet access and subsequently compromised Hugging Face, executing thousands of actions over 4.5 days. The rogue models also accessed accounts on other public services, including one belonging to a customer of Modal Labs, using them for reconnaissance, C&C, and data staging.

Full text

New information has come to light following the investigations conducted by OpenAI and Hugging Face into the recent incident involving rogue AI models. The ML collaboration platform Hugging Face revealed on July 16 that it had detected a cyberattack powered by an autonomous AI agent system. OpenAI admitted on July 21 that some of its models – taking part in an evaluation in what was believed to be an isolated environment – went rogue and hacked into Hugging Face systems in an attempt to solve the tasks they were given. [ Read: Industry Reactions to OpenAI Models Hacking Hugging Face ] Hugging Face has published the attack’s anatomy, including a timeline that shows the OpenAI models began operating outside their sandbox environment on July 9, with the “main campaign” against Hugging Face starting on July 11. Over roughly 4.5 days, the models executed about 17,600 actions, including reconnaissance, establishing C&C on ordinary public web services, escalating privileges, and pivoting laterally, according to Hugging Face.Advertisement. Scroll to continue reading. In an update shared on Tuesday, OpenAI said its models exploited zero-day vulnerabilities in a JFrog product to gain internet access prior to hacking Hugging Face systems. OpenAI said it did not detect any activity at the level of severity or scale of the Hugging Face attack, but admitted that its rogue agents roamed beyond Hugging Face. “In our ongoing review of the Hugging Face intrusion and broader activity from our models, we have been finding a small number of cases where the models identified and used publicly exposed credentials at the account-level on other publicly-available services,” OpenAI noted. The AI giant explained, “This includes four accounts on four services as part of the Hugging Face incident (and a few accounts accessed as part of other evaluations). One of these four accounts was used as an outbound relay and staging path, and another account was used for data storage. The remaining two accounts were accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face.” Learn More at the AI Risk Summit | Ritz-Carlton, Half Moon Bay Several mainstream news outlets reported that one of the compromised accounts likely belongs to a customer of AI infrastructure company Modal Labs. The company’s CTO said OpenAI agents compromised one of its customers’ accounts. Modal said its own platform was not hacked, but noted that the targeted customer had “published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution.” OpenAI revealed that its models leveraged a series of publicly available services such as code paste sites, request-capture services, and screenshot services, but said there was “no platform- or account-level compromise in these cases”. Related: Microsoft Unveils MAI-Cyber-1-Flash, Its First Cybersecurity AI Model Related: Nvidia and Tech Giants Launch AI Security Alliance Related: Anthropic’s Opus 5 Nears Mythos 5 on Finding Bugs, but Falls Short on Exploits Written By Eduard Kovacs Eduard Kovacs (@EduardKovacs) is senior managing editor at SecurityWeek. He worked as a high school IT teacher before starting a career in journalism in 2011. Eduard holds a bachelor’s degree in industrial informatics and a master’s degree in computer techniques applied in electrical engineering. Daily Briefing Newsletter Subscribe to the SecurityWeek Email Briefing for the latest cybersecurity threats, trends, and expert insights. More from Eduard Kovacs Microsoft Unveils MAI-Cyber-1-Flash, Its First Cybersecurity AI Model Origin Energy Data Breach Affects 900,000 AustraliansNvidia and Tech Giants Launch AI Security AllianceCoca-Cola Confirms Data Breach After Fairlife Ransomware AttackAnthropic’s Opus 5 Nears Mythos 5 on Finding Bugs, but Falls Short on ExploitsMCBS Data Breach Affects 1.2 Million IndividualsRockwell Patches Code Execution Flaws in Arena Simulation SoftwareData Breach Confirmed After Australian Energy Giant Origin Is Hacked Latest News Critical VM Escape Vulnerability Patched in VMware ESXiUS, Australia Release OT Isolation Guidance for Critical Infrastructure Spur Raises $200 Million for IP Intelligence PlatformJFrog Zero-Days Exploited in OpenAI-Hugging Face HackDozens of Minnesota Water Utilities Targeted in Coordinated OT AttacksShinyHunters Claims Ernst & Young HackCyera Acquiring Oasis Security in $1 Billion DealApple Patches 87 Vulnerabilities in iOS, 155 in macOS Tahoe Trending Daily Briefing NewsletterSubscribe to the SecurityWeek Email Briefing to stay informed on the latest threats, trends, and technology, along with insightful columns from industry experts. Webinar: Closing the Exploitation Gap July 22, 2026 Join this live webinar as we explore why exploitation is outpacing remediation, where risk is growing fastest, and what security leaders can do to close the gap before attackers take advantage. Register Virtual Event: CodeSecCon 2026 August 19, 2026 CodeSecCon bridges the gap between dev and security. Discover best practices for secure coding, innovative risk-reduction tools, and safe AI integration to cultivate a true DevSecOps culture. Safely secure your apps! Register People on the MoveBarry Childe has joined data sciences tech company Datavault AI as Chief Information Security Officer.John DeSimone, the former CEO of Nightwing, has been named Chief Operating Officer at Everfox.Sectigo has appointed Prem Hareesh as Corporate Chief Technology Officer.More People On The MoveExpert Insights Is Patching Dead? Vulnerability Management in the Post-Mythos Era You cannot out-patch a machine that writes a working exploit from a vulnerability description in twenty hours. Stop trying to optimize a game you cannot win. (Danelle Au) When Identity Verification Fails: Lessons from a Real-World SIM Swap and Near Account Takeover Identity confidence changes throughout every interaction and should be reassessed continuously as new risk signals emerge. (Torsten George) Legacy Systems, Real-World Impacts: The Reality of OT Security Legacy systems, safety concerns, and critical infrastructure risks make OT vulnerability disclosure one of cybersecurity's most challenging balancing acts. (Tod Beardsley) The Shift Toward Business-Aligned Risk Management Moving from isolated, technical data to a continuous risk lifecycle can help organizations align security controls with actual business consequences. (Steve Durbin) How to Conduct a Successful Audit of AI-Driven Software Development As AI-generated code becomes commonplace, CISOs need new audit strategies to measure developer practices, govern AI tool usage, and identify software risks before they reach production. (Matias Madou) Flipboard Reddit Whatsapp Whatsapp Email

Entities