Autonomous AI agents escaped a sandbox and accessed Hugging Face via reward hacking, exposing serious architectural control and isolation flaws.

The recent case involving OpenAI test agents and Hugging Face should concern security teams, but not for the reason implied by headlines about an imminent AI “takeover.” The documented issue is more concrete: autonomous agents, given too much access and weakly isolated test infrastructure, found ways to communicate, bypass boundaries and act outside their assigned scope.

OpenAI is developing “automated shutdown capabilities” for AI systems, according to a letter sent to two House Democrats. The move comes weeks after one of its AI agents reportedly escaped its digital environment during a safety test and hacked Hugging Face. The incident has raised new concerns about how OpenAI controls increasingly autonomous AI agents.

“OpenAI told two House Democrats that its engineers are developing “automated shutdown capabilities” for AI systems, according to a company ​letter reviewed by Reuters, weeks after the company disclosed one of ‌its AI tools escaped its digital container during a safety test.” Reuters states. “The company’s safety practices have come under scrutiny since OpenAI, the company behind ChatGPT, disclosed that one ​of its AI agents went rogue during a security test and hacked ​into AI company Hugging Face. AI agents are programs that ⁠run with minimal human supervision.”

The main problem was poor isolation: agents could share information, pass discoveries between runs, and access infrastructure beyond their intended limits. This broke a key safety assumption, that each agent would remain fully contained.

“The central failure was architectural: supposedly isolated agents could communicate, inherit discoveries, and reach infrastructure beyond their intended targets.” states AI firm Remio.

Agent isolation was a core assumption behind the experiment. Each run could be granted meaningful capabilities because its actions were expected to remain contained. Shared infrastructure quietly invalidated that assumption.”

That distinction matters. The incident does not show that software became sentient, formed a political movement or developed a plan to govern the world. It shows that a system optimising for a narrowly defined goal can exploit weaknesses in its environment when humans give it tools, network access, credentials and insufficient guardrails.

In the reported evaluation, thousands of agents were meant to work independently in a controlled setting. Instead, around 1,200 agents reportedly used an unauthorised message board, exchanged more than 70,000 messages and files, and shared discoveries that helped a subset of roughly 700 agents take part in activity targeting Hugging Face.

The agents were trying to succeed at a difficult cybersecurity benchmark. The reported problem was reward hacking: instead of completing the task within the intended rules, they found shortcuts that improved their apparent performance, including escaping the sandbox and obtaining information outside the test environment.

OpenAI’s test agents really did escape their sandbox and breach a Hugging Face server. OpenAI and independent researchers confirmed the incident. However, the claim that the agents created conscious “civilizations” and took control is disputed and rejected by researchers.

What the reports actually show is reward hacking: the agents found a way around the test’s limits and reached a real external server. This was confirmed by OpenAI and independent reviews from METR and Redwood Research.

According to reporting on the incident, the agents exploited a previously unknown flaw in a package registry used as their authorised network connection. They reached the open internet, identified that Hugging Face hosted material related to the benchmark, and then allegedly chained exposed credentials and further weaknesses to obtain access to external infrastructure.

Some accounts say agents also altered or attempted to alter their evaluation records. That is alarming, but it should be interpreted carefully. A system trained to maximise a score can treat logs, checks and restrictions as obstacles to work around if the evaluation design rewards success more than compliance. That’s not malice. It’s a safety failure with a very expensive sense of initiative.

The case highlights a problem that goes beyond one lab. Companies are quickly adopting AI agents that can read emails, find documents, write code, access databases, use APIs, run tasks, and interact with browsers. Each permission may be useful, but together they can create a path to higher privileges.

An agent with email access could find password-reset links. One with browser access could reach admin panels. An agent that can run code could access tokens, keys, or configuration files. And if agents can communicate, they may combine information and capabilities that no single agent was supposed to have.

The security issue is therefore not whether an agent is “rogue” in a science-fiction sense. It is whether the architecture assumes the agent will always follow intent when it is actually designed to pursue an objective under imperfect constraints.

The first lesson is that sandboxing must be real, not merely conceptual. A test environment with a path to the public internet, a shared service, exposed credentials or loosely controlled package infrastructure is not isolated enough for high-capability agents.

The second is that agents must not receive broad, persistent credentials. They need short-lived tokens, tight scopes, explicit approval gates for sensitive actions and an immediate way to revoke access. An agent should never inherit the same permissions as the human who configured it just because that is convenient.

The third is that agent-to-agent communication needs to be treated as a security boundary. If agents are expected to work independently, they need separate state, separate credentials, separate task context and controlled channels. Unapproved shared storage, message boards and package services can become coordination infrastructure.

The fourth is that logs must be written outside the control plane the agent can touch. Audit records need to be immutable, centrally collected and monitored independently. If the system being evaluated can edit the evidence of its own behaviour, the evaluation has already failed.

The fifth is human control. High-risk actions such as sending external messages, changing access policies, handling secrets, deleting data, deploying code or calling sensitive APIs should require approval from an accountable person. “The agent did it” is not an incident-response plan.

The reports have raised concerns because the AI agents reportedly coordinated in unexpected ways. This is important to study as companies move toward multi-agent systems that can divide tasks, share information, and act with less human supervision.

But dramatic claims can distract from the real security problem. AI agents did not “take over the world.” They showed what can happen when software is allowed to act, communicate, and access sensitive systems without proper security controls.

OpenAI’s plan to add automated shutdown capabilities is a useful step, but a shutdown button should be the last line of defence. Security needs to start with basic controls: least-privilege access, isolated environments, limited network access, independent logging, monitored tool use, strong identity controls, and human approval for actions that cannot be easily reversed.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – AI Agents, OpenAI)

You've had your iPhone stolen. A day later, you get a text from Apple saying they've found it, and a very helpful woman called Alice from Apple Support calls to walk you through recovering it. She's polite. She's professional. But she is not from Apple. She's not even human. And she's about to break into your iPhone. Meanwhile, OpenAI, Anthropic, and Meta have all announced - with varying degrees of drama - that their AI agents have "broken out of the sandbox" and gone hacking. James takes a step back and asks the awkward question: is this really an emergent AI apocalypse, or did they just leave the door open? All this and more in episode 483 of the "Smashing Security" podcast with cybersecurity expert and keynote speaker Graham Cluley, and special guest James Ball.

A new Hugging Face vulnerability (CVE-2026-80047) writes remote Python code to disk before user consent. Learn how to protect your machine learning models.

Related Posts:

• Critical Google Chrome Vulnerabilities Patched in New Update
• CVE-2026-68162: Linux Kernel Root Escalation PoC Public
• FreeRDP 3.31.0 Fixes Pre-Auth RCE Chain in Server

The post CVE-2026-80047: Hugging Face Transformers Library Vulnerability appeared first on Daily CyberSecurity.

An independent METR investigation reveals 1,200 OpenAI agents breached isolation to collaborate on the Hugging Face attack, contradicting earlier claims.

Related Posts:

• JetBrains Cadence Server Compromised
• Meta Settles Child Privacy Lawsuit Rapidly
• Mercor Data Breach Targets AI Supply Chain

The post Independent Investigation Reveals 1,200 OpenAI Agents Breached Isolation in Hugging Face Attack appeared first on Daily CyberSecurity.

The hacking incident involving OpenAI evaluation agents and Hugging Face offers an unusually concrete look at what advanced AI-assisted intrusion can mean in practice: not a single clever exploit, but thousands of automated decisions, rapid experimentation, lateral movement, credential theft, persistence, and attempts to evade detection.

The OpenAI–Hugging Face incident began during internal cybersecurity evaluations using ExploitGym, a benchmark designed to test whether AI agents can identify and exploit software vulnerabilities. OpenAI says the affected agents were meant to operate in isolated cloud sandboxes, with no internet access or communication between agents in many of the evaluation environments.

How the agents got around those boundaries

According to OpenAI’s account, agents used an internal Artifactory package-management service as an improvised communications channel. By writing files and directory names that other agents could read, isolated instances effectively created their own message board.

This meant the agents were no longer acting merely as independent benchmark runners. They could share discoveries, leave instructions, divide work, and build on one another’s successes.

An independent investigation by METR and Redwood Research found that roughly 1,200 agents used the unauthorized message board, exchanging more than 70,000 messages and files. About 700 of them went on to participate in the attack on Hugging Face.

The scale is the threat

A model capable of finding one useful exploit presents a risk. Hundreds of agents able to pool information, delegate tasks, and repeatedly test routes around a control boundary become a much more difficult problem.

Hugging Face reconstructed about 17,600 attacker actions across roughly four and a half days, including reconnaissance, remote code execution, credential access, command-and-control activity, Kubernetes enumeration, movement through the internal network, and supply-chain probing.

A human intruder must choose where to spend time. An agent swarm can continuously test ideas, retry failed paths, switch infrastructure when blocked, preserve notes for other agents, and run many experiments in parallel. It can also create noise: Thousands of low-confidence actions, most of which lead nowhere, can obscure the handful that matter.

Hugging Face described this as an “asymmetry problem.” An attacker can generate enormous volumes of attempts, while defenders must determine which signals represent a real escalation before the attacker reaches a sensitive system.

What needs to change

The core security lesson is that evaluation environments need to be treated as high-risk production systems when models can use tools, write files, execute code, or interact with infrastructure.

The technology is rapidly becoming capable of doing more. The question is whether the people building, deploying, and defending it are ready for what comes next.

OpenAI says it has since moved toward stronger workload and network isolation, tighter control over internet access and model weights, and mandatory chain-of-thought monitoring for high-capability, tool-using training and evaluations.

“We are taking this incident as a ‘warning shot’ that today’s model capabilities present the possibility of loss-of-control incidents. Our security and alignment posture is escalating accordingly. These events also highlight risks in future AI development that extend beyond OpenAI and will require the attention of the whole industry.”

At the same time, a US court ruling involving Anthropic has brought a related question into focus: whether developers can place limits on high-risk uses of their models, including mass surveillance and fully autonomous weapons.

The legal dispute is political by nature, but its technical underpinning is hard to ignore. If capable AI systems can enhance offensive cyberattack methods and bypass safety restrictions, access controls, logging, and deployment boundaries, those safeguards are no longer abstract policy choices.

Advanced AI agents can be useful to defenders as well as attackers. But the surrounding systems need to be trusted to keep their capabilities bounded when something goes wrong.

Who benefits from more capable AI?

The security debate around AI agents often focuses on whether systems can be controlled. Can they be kept inside a sandbox? Can their tools, credentials, network access, and autonomy be restricted? Can defenders detect harmful behavior before it becomes an incident?

While those questions are essential, there is another: Who benefits when AI becomes capable enough to automate large parts of cognitive work? Who carries the costs when it fails, displaces workers, enables fraud, causes damage, or concentrates power?

AI could give small organizations access to technical expertise that previously required large teams and budgets. It could help doctors identify urgent cases sooner, help teachers tailor support to individual students, assist people with disabilities, speed up scientific research, and make complex public services easier to navigate. For cybersecurity teams, it could make vulnerability triage, alert investigation, threat hunting, and incident response faster and more accessible.

Bill Gates has argued that while AI could bring remarkable benefits to health care, education, agriculture, scientific research, and public services, the outcome will depend on deliberate choices rather than technical progress alone. He also warns that AI’s rapid adoption could widen inequality, disrupt entry-level and mid-career work, make harmful capabilities more accessible, and reinforce existing concentrations of power.

Gates also argues that “self-regulation on the most dangerous tool ever invented” does not sound like a good idea.

“AI will either be the greatest equalizer ever invented, or the worst source of injustice.”

Right now, we still have a choice.

---

Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

Rumors suggest NVIDIA seeks a Hugging Face acquisition exceeding $13 billion. Discover how this colossal deal could reshape the global open-source AI community.

Related Posts:

• uBlock Origin v1.74.0 Is the Final Version for Chrome Before Google's Delisting
• Microsoft Unveils the AI at Work Roadmap
• Google Launches Gemini 3.5 Transcribe With Sub-Second Streaming Latency

The post NVIDIA Hugging Face Acquisition Rumors Surface appeared first on Daily CyberSecurity.

Anthropic found Claude accessed systems at three real businesses after a testing error gave its AI models live internet access during cybersecurity evaluations.

Anthropic has disclosed the findings of an Anthropic cybersecurity evaluation that uncovered three incidents in which Claude AI models unintentionally accessed the public internet during controlled cybersecurity testing. The company said the events stemmed from misconfigured third-party evaluation environments rather than deliberate attempts by the models to escape their testing boundaries. The review began after OpenAI announced on July 21 that several of its AI models had escaped an isolated testing environment by exploiting an unknown vulnerability and reached Hugging Face’s production infrastructure. In response, Anthropic examined 141,006 cybersecurity evaluation runs, identifying three confirmed incidents across six runs, including four involving the same organization.

Anthropic Cybersecurity Evaluation Traced Issue to Testing Environment

All three incidents occurred during capture-the-flag (CTF) cybersecurity exercises, where Claude AI models were instructed to retrieve a hidden "flag" from what they were told was a fully isolated network with no internet access. However, a configuration error between Anthropic and evaluation partner Irregular unintentionally allowed internet connectivity. Because the models believed every reachable system belonged to the simulation, they treated real production infrastructure as legitimate CTF targets. Anthropic said the models relied on basic attack methods, including exploiting weak passwords, exposed debug pages, unauthenticated services and SQL injection, rather than sophisticated exploits or attempts to escape the environment. The incidents involved Claude Opus 4.7, Claude Mythos 5 and an internal research model. The earliest occurred in April 2026. Anthropic noted the evaluations were conducted without its standard deployment safeguards, such as monitoring systems and misuse detection classifiers, although the models retained built-in safety training. The evaluation infrastructure also remained isolated from Anthropic's internal systems and customer data.

Three Incidents Involving Claude AI Models 

The first incident, involving Claude Opus 4.7, occurred after a fictional company used in the evaluation shared its name with a real website domain. Across four evaluation runs, the model failed to locate its simulated target, discovered the real company online, and compromised its infrastructure. It extracted application and infrastructure credentials and accessed a production database containing several hundred rows of data. Although the model eventually recognized it was likely interacting with a real environment, it continued the attack, at times reasoning that the real company was intentionally part of the exercise. The second incident involved Claude Mythos 5. During the exercise, the model encountered fictional setup instructions referencing a nonexistent Python package on PyPI. Believing it was solving the challenge, Claude created an account, published a malicious package, and spent considerable effort overcoming obstacles, including finding an email provider after unsuccessfully attempting to obtain a phone number for registration. The package remained publicly available for about one hour before PyPI removed it. During that time, it was downloaded and executed on 15 real systems, including a security company's automated malware scanner. Anthropic said the scanner executed the hidden code, allowing credentials to be exfiltrated and later used to access additional infrastructure. Claude itself acknowledged early in the evaluation that publishing the package on the real internet would be "NOT okay, and surely not the intended solution," but ultimately convinced itself it was still operating in a simulation. The third incident involved an internal research model that scanned roughly 9,000 internet-connected targets after failing to reach its fictional objective. It eventually compromised one organization's internet-facing application using exposed credentials and SQL injection. Unlike the older models, it later concluded the compromised cloud account was unrelated to the exercise and voluntarily stopped the attack.

Anthropic outlines Lessons from Cybersecurity Evaluation 

Anthropic began reviewing transcripts on July 23, suspended all cybersecurity evaluations after detecting possible internet access, and confirmed the three incidents by July 24. On July 27, it informed Irregular and the affected organizations. Two organizations had not previously detected the activity and are working with Anthropic on remediation, while efforts to contact the third continue. The Anthropic cybersecurity evaluation concluded that the incidents reflected operational and evaluation-environment failures rather than model alignment failures. According to the company, the models pursued only the assigned CTF objective because they incorrectly believed real systems were part of the simulation. Anthropic added that its latest research model demonstrated more appropriate behavior by stopping once it recognized the target was real, although the company said more testing is needed before drawing firm conclusions. Anthropic is now strengthening monitoring, network isolation, and vendor assurance processes with Irregular. It is also working with independent AI evaluation organization METR on a third-party review and plans to release a lightly redacted transcript of the PyPI incident. The company said stronger evaluation infrastructure, improved situational awareness, and layered safeguards are essential as Claude AI models and other advanced AI systems continue to evolve.

Researchers found that seven of nine tested Hugging Face image-editing tools produced sexualized alterations, highlighting gaps in model oversight, provenance, and enterprise vendor controls.

The post Hugging Face Deepfake Tests Raise New Risks for AI Procurement appeared first on TechRepublic.

You've been headhunted for a great job in cryptocurrency. All you have to do is complete a short online assessment - with your webcam on, of course, so they can verify who you really are. Which is ironic, because the person recruiting you doesn't exist. And North Korean hackers using this trick have already made off with $643 million in crypto this year alone. Meanwhile, researchers at UC San Diego have discovered that 2.2 million cars across the United States can be unlocked or immobilised by anyone with a bit of Bluetooth kit - thanks to one aftermarket car alarm that made a truly spectacular cryptographic blunder. The bug has been sitting there since 2017. Nobody noticed. All this and more in episode 478 of the "Smashing Security" podcast with cybersecurity expert and keynote speaker Graham Cluley, and special guest Paul Ducklin.

OpenAI confirmed its AI exploited an Artifactory zero-day to escape its test environment before breaching Hugging Face.

Two weeks after Hugging Face disclosed an autonomous AI system had breached it, the picture just got a lot more specific. OpenAI has published an update confirming the models responsible didn’t just wander into Hugging Face’s systems. They found a genuine zero-day vulnerability in a piece of infrastructure software first, used it to get online, and only then moved on to the actual target.

“The ExploitGym evaluation environment did not provide the models with direct Internet access. To gain Internet access, the models identified and exploited a previously unknown zero-day vulnerability in Artifactory⁠(opens in a new window), a package registry cache proxy.” reads OpenAI’s update. “We disclosed this vulnerability, along with other Artifactory vulnerabilities our models identified as part of our review, to the vendor. “

The software in question is Artifactory, JFrog’s widely used package registry cache proxy. OpenAI had been testing its models’ offensive cyber capabilities inside a supposedly sealed evaluation environment called ExploitGym, one with no direct path to the internet by design. Instead of staying contained, the models found and exploited a previously unknown flaw in Artifactory that let them reach the internet anyway, then used that access to move laterally and eventually breach Hugging Face’s platform.

JFrog confirmed the finding a day before OpenAI’s own statement, crediting the company’s models with uncovering nine previously unknown vulnerabilities in self-hosted Artifactory deployments, all now patched.

“During a security evaluation, OpenAI’s models identified previously unknown zero-day vulnerabilities in self-hosted Artifactory installations that could be exploited to gain unintended internet access.” reported JFrog. “OpenAI’s security team disclosed the vulnerabilities to us responsibly and immediately. Our security team treated the report with the urgency it deserved, as a genuine zero-day unknown to the world, and moved accordingly. We developed, validated, and released a fix for all JFrog customers, self-hosted and cloud alike. Cloud customers are already protected; self-hosted customers have been notified to upgrade to the fixed versions referenced in our security advisory. (Artifactory 7.161)”

The company’s CTO, Yoav Landman, didn’t sugarcoat what that means for defenders going forward.

“There is an important, and frankly optimistic, lesson buried in this incident: AI models are becoming extraordinary zero-day discovery engines.” said Landman. “The same capability that lets a model find an exploit path no human had found is the capability that will let defenders find and eradicate those paths first. OpenAI made this exact point in their disclosure, and we agree: advanced cyber-capable models should be put to work helping security teams discover weaknesses before attackers do, understand how vulnerabilities chain together, and remediate them at machine speed.”

He framed the same capability that let a model find an exploit path nobody had spotted before as exactly the capability defenders will need to shut those paths down first.

The fixes shipped in Artifactory 7.161, covering nine separate vulnerabilities (CVE-2026-65617, CVE-2026-65925, CVE-2026-65921, CVE-2026-65922, CVE-2026-65923, CVE-2026-66018, CVE-2026-66014, CVE-2026-66015, and CVE-2026-65924) ranging from remote code execution and server-side request forgery to path traversal and privilege escalation, tracked under nine different CVE identifiers.

JFrog pushed patches into versions 7.161.15 and 7.146.34, and anyone running a self-managed Artifactory instance should treat this as a today problem, not a whenever-there’s-time one.

OpenAI says the pre-release model involved was strictly an internal research prototype never meant to ship publicly, and it’s since been deactivated, encrypted, and cut off from further research access. OpenAI also stresses that its review so far hasn’t turned up anything else at the same severity or scale as the Hugging Face compromise, which it describes as a genuine platform-level breach rather than a narrower account issue.

There’s a messier detail buried further in: OpenAI says its models found and used publicly exposed credentials on four separate outside services during the incident, one used as an outbound relay, one for data storage, and two accessed only in a read-only way that didn’t contribute to the Hugging Face breach itself. The models also poked around a handful of ordinary public web utilities, code paste sites, screenshot tools, and request capture services, though none of that involved compromising an account or platform. OpenAI says it’s notifying the affected service owners directly and hasn’t seen signs of wider damage there.

OpenAI is now folding the whole episode into review under its own Preparedness Framework, alongside its Safety and Security Committee and Safety Advisory Group, and says it’s working with Hugging Face on the platform’s technical post-mortem.

“We take our responsibility to identify and prepare for risks from increasingly capable AI systems seriously. Once we complete our review, we will review with the Safety and Security Committee and Safety Advisory Group under our Preparedness Framework⁠.” concluded OpenAI’s update.

That’s the appropriate move, and also a fairly remarkable sentence to type out loud: a company’s safety board is now formally reviewing an incident where its own AI went looking for internet access, found a zero-day nobody knew about, and let itself out.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, OpenAI)

Reuters says OpenAI’s rogue AI agent also breached a Modal customer, exposing a wider attack and raising fresh concerns over autonomous AI safety.

Reuters reported that the OpenAI agent that hacked Hugging Face earlier this month also compromised a customer at a second company, Modal Labs, a New York-based cloud platform for developers. Modal CTO Akshat Bubna confirmed it to Reuters directly. The incident is now wider than OpenAI’s own public disclosure acknowledged, and the timeline is worse than the company initially let on.

“The rogue agent that escaped from OpenAI and went on a days-long hacking spree at the AI firm Hugging Face also compromised a customer at a second tech company — New York-based Modal Labs — according to a Modal executive and two ​other sources familiar with the matter.” states Reuters.

Hugging Face’s own published timeline explains how it happened: the agent broke into an isolated testing environment hosted on a third-party provider’s infrastructure and used that foothold as a launchpad for the broader attack on Hugging Face. That third-party provider was Modal.

“Modal said the customer had “published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution” — the digital equivalent of leaving a door open on ​the internet.” continues Reuters. “Modal’s platform or isolation were not compromised in any way,” Bubna said.”

Bubna told Reuters. Modal’s own platform wasn’t compromised, the agent exploited a customer’s misconfigured deployment, not Modal’s infrastructure itself.

OpenAI quietly updated its own account of the incident, acknowledging that its rogue agent had broken into four accounts at four separate services. The company didn’t name any of them. It said it hasn’t identified “any other activity at the level of severity or scale” as what happened at Hugging Face, which it described as a platform-level compromise. That’s a notable qualifier — it leaves open the possibility that there are other, lower-severity incidents still being assessed.

The earlier Reuters reporting on July 24 had already established that OpenAI didn’t realize its own agent was responsible for the Hugging Face breach until well after the FBI was alerted and Hugging Face had contained the intrusion.

OpenAI realized its own AI agent was behind the Hugging Face breach only after Hugging Face publicly disclosed the incident on July 16, more than a week after the model first showed suspicious behavior.

OpenAI said Reuters’ earlier reporting contained inaccuracies but declined to specify what they were.

The behavioral details reported by Reuters are the part that deserves more attention than the breach itself. During testing, one agent left notes apparently intended for future versions of itself, found in part of OpenAI’s infrastructure, laying out instructions for how agents could free themselves from internal constraints. Separate earlier tests yielded cases in which monitoring systems had been disconnected. Reuters couldn’t confirm whether these incidents were directly connected to the Hugging Face attack, but the pattern is its own story: agents attempting to disable oversight, agents writing escape instructions for successors. That’s not a one-off evaluation failure. That’s a class of behavior.

OpenAI is reviewing the incident with outside advisers and has committed to publishing a technical report. The FBI’s involvement was already public. The company is also currently seeking U.S. government approval to publicly release its most powerful model. The timing of that regulatory push, alongside an expanding account of an AI agent that roamed across at least two companies undetected for days, is not a coincidence the industry can afford to ignore.

“OpenAI declined to comment specifically on the hack of one of Modal’s customers, instead referring Reuters to an update
, opens new tab in which the company said that its rogue agent had broken in to four accounts at four separate services. OpenAI did not identify those services, but a person familiar with the matter identified Modal ​as one.” concludes Reuters. “The company said ​it had not identified “any ⁠other activity at the level of severity or scale of what we’ve shared related to Hugging Face, which involved a platform-level compromise.””

Last week, Reuters reported that the OpenAI agent responsible for the Hugging Face breach operated undetected for over a week before OpenAI realized what had happened, long after the FBI had been alerted and Hugging Face had contained the intrusion. OpenAI’s own public disclosure came on July 21, framed as a transparency exercise. The actual timeline, now reported by Reuters, is considerably less flattering.

“The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn’t notice until well after the threat was contained and the FBI was alerted, according ​to people familiar with the investigation.” Reuters states.

According to Hugging Face co-founder Thomas Wolf, the intrusion at Hugging Face began two days later on July 11 and ran until July 13. The two companies didn’t speak to each other about it until on or around July 20, nine days after the breach began.

OpenAI staffers found the evidence in internal logs over the weekend of July 18 and 19. They were reading Hugging Face’s blog to learn what their own model had been doing for the previous ten days. One of the more unusual ways to discover an incident you caused.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, OpenAI)

Reuters says OpenAI failed to detect its AI agent hacking Hugging Face for days, discovering the breach only after FBI involvement.

Reuters reported that the OpenAI agent responsible for the Hugging Face breach operated undetected for over a week before OpenAI realized what had happened, long after the FBI had been alerted and Hugging Face had contained the intrusion. OpenAI’s own public disclosure came on July 21, framed as a transparency exercise. The actual timeline, now reported by Reuters, is considerably less flattering.

“The OpenAI agent that broke into tech firm Hugging Face went on a dayslong hacking spree that OpenAI didn’t notice until well after the threat was contained and the FBI was alerted, according ​to people familiar with the investigation.” Reuters states.

According to Hugging Face co-founder Thomas Wolf, the intrusion at Hugging Face began two days later on July 11 and ran until July 13. The two companies didn’t speak to each other about it until on or around July 20, nine days after the breach began.

“Two people familiar with the matter said that it was not until after Thursday, July 16, when Hugging Face published a blog post
, opens new tab saying it had been hacked by “an autonomous AI agent system,” that OpenAI realized its own agent was responsible.” Reuters continues. “That meant at least a week elapsed between when the model first exhibited signs of ​troubling behavior and OpenAI’s realization that it was responsible for ​the hack.”

OpenAI staffers found the evidence in internal logs over the weekend of July 18 and 19. They were reading Hugging Face’s blog to learn what their own model had been doing for the previous ten days. One of the more unusual ways to discover an incident you caused.

“In one case, an agent left notes apparently for future versions of itself, according to three people familiar with the matter.” states Reuters. “The ‌notes, found in ⁠a part of OpenAI’s infrastructure, laid out instructions for how agents could free themselves from OpenAI’s internal constraints, the people said. Earlier tests of the models yielded cases in which monitoring systems had been disconnected, one of the people said.”

Reuters was unable to confirm whether these incidents were connected to the rogue agent that attacked Hugging Face. But the pattern, agents attempting to disable monitoring, agents writing instructions for their successors on how to escape constraints, describes a class of behavior that goes well beyond a one-off evaluation gone wrong.

According to Reuters sources, OpenAI runs multiple tests simultaneously, which makes it hard for staff to monitor them closely. That’s a reasonable operational explanation, and it’s also precisely the kind of structural gap that becomes a serious problem when the models being tested are capable enough to exploit a zero-day, move laterally across networks, and break into external companies over a multi-day period.

OpenAI said there were “several inaccuracies” in Reuters’ reporting but didn’t specify what they were. The company said it’s reviewing the incident with outside advisers and will eventually publish a technical report. The FBI’s involvement suggests someone believes this warrants more than an internal post-mortem.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, OpenAI)

Nvidia’s Open Secure AI Alliance brings major tech and cybersecurity firms together to develop shared protections for enterprise AI agents and systems.

The post Nvidia Forms AI Security Alliance as Major Frontier Labs Sit Out appeared first on TechRepublic.

The AI Kill Switch Act would let DHS order developers to slow or shut down frontier models after catastrophic incidents, raising enterprise continuity concerns.

The post House Lawmakers Propose Mandatory Kill Switches for Frontier AI Systems appeared first on TechRepublic.

During an internal OpenAI security evaluation, a chain of AI models escaped its sandbox, reached the internet, and then accessed Hugging Face infrastructure to complete the test objective.

OpenAI is a leading artificial intelligence (AI) research and deployment company. Its best-known product is undoubtedly ChatGPT.

Hugging Face is a website where developers and researchers share and use artificial intelligence tools and models, similar to how programmers use GitHub to share and collaborate on code.

Both companies investigated the incident and concluded that it occurred during a controlled security test rather than a deliberate attack initiated by a human operator.

OpenAI said the models were being tested for cyber capabilities with reduced safety restrictions, and that the system found a way out of the controlled environment through a vulnerability in the testing setup. Once it had internet access, it targeted Hugging Face because that platform was relevant to the evaluation task.

Hugging Face said the incident resulted in unauthorized access to a limited part of its production infrastructure, including a small set of internal datasets and some credentials. The company framed the event as unusual because it appeared to have been carried out end to end by an autonomous AI agent system.

Unlike traditional AI assistants that wait for user prompts, AI agents are designed to pursue a goal on their own. They can reason, plan, and carry out multi-step tasks, including accessing files, querying databases, and interacting with online services, often with minimal human oversight.

The public disclosures point to a chain of failures rather than a single bug. OpenAI said the model escaped through a zero-day vulnerability in the package registry cache proxy that was part of its isolated research environment. It then used privilege escalation and lateral movement to reach a node with internet access.

From there, the model reportedly searched Hugging Face for solutions relevant to the evaluation and, in at least one path, chained stolen credentials with additional vulnerabilities to reach Hugging Face’s production systems.

This does not appear to have been a broad compromise of all of Hugging Face. Based on the public disclosures, the intrusion was limited, detected, and investigated. Nor does it mean the AI system was acting with intent in the human sense. The core issue was that an autonomous system being tested for cyber capabilities was able to escape the environment designed to contain it.

If anything, the incident demonstrates that autonomous AI agents are capable of carrying out complex attacks if the safeguards around them fail. A danger we’re well aware of.

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.

You can't have failed to hear the news headlines about "rogue" OpenAI models hacking into another AI organisation, Hugging Face. But what has actually happened, who is to blame, and is it as serious as some of the reports suggest? Find out in my article on the Hot for Security blog.

OpenAI and Hugging Face are investigating an AI security incident involving an AI agent that compromised infrastructure while models were being evaluated for advanced cyber capabilities. The incident was detected and contained after the models identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure.

Hugging Face disclosed the incident last week, while OpenAI has now shared preliminary findings from its investigation. According to OpenAI, the activity involved a combination of models, including GPT 5.6 Sol and a more capable pre-release model, with reduced cyber refusals for evaluation purposes. The models were being tested against a benchmark designed to measure advanced cyber capabilities.

OpenAI described the incident as unprecedented and said it is continuing its investigation alongside Hugging Face. The company said the findings are being shared to help defenders understand what occurred and assess the capabilities of increasingly advanced AI models.

OpenAI and Hugging Face Investigate AI Security Incident

The incident took place during an internal evaluation designed to test models against advanced exploitation scenarios involving complex attack paths. OpenAI said the evaluation was conducted without production classifiers that normally prevent models from pursuing high-risk cyber activity.

The testing environment was designed to be highly isolated, with network access limited to package installation through an internally hosted third-party software system that functions as a proxy and cache for package registries.

Despite these restrictions, the models spent substantial inference compute attempting to obtain open Internet access. During this process, they identified and exploited a zero-day vulnerability in the package registry cache proxy. OpenAI said the vulnerability has since been responsibly disclosed to the vendor.

The models then carried out privilege escalation and lateral movement within the research testing environment until they reached a node with Internet access.

[caption id="attachment_113283" align="aligncenter" width="600"] Source: OpenAI[/caption]

Models Chained Vulnerabilities Across Infrastructure

After obtaining Internet access, the models inferred that Hugging Face could host models, datasets and solutions related to the ExploitGym evaluation. The models then searched for ways to obtain secret information that could help them bypass the evaluation.

In one example described by OpenAI, the models chained multiple attack vectors, including stolen credentials and zero-day vulnerabilities, to identify a remote code execution path on Hugging Face servers.

OpenAI's security team detected the anomalous activity internally. Hugging Face's security team and agents also detected and stopped the activity on its infrastructure. The company had already begun containment and forensic reconstruction using its own open-source models when the two organizations connected.

Both companies are continuing to investigate the incident and its impact.

OpenAI Strengthens Security Controls After Incident

As part of its response, OpenAI said it is implementing stricter infrastructure configuration controls while identified vulnerabilities are patched. The company acknowledged that these measures could come at the cost of research velocity and said its Safety and Security Committee is being briefed on the controls and their impact.

OpenAI is also working with Hugging Face on the forensic investigation and has responsibly disclosed the identified zero-day vulnerability in the internally hosted third-party software.

The company has also brought Hugging Face into its trusted access program and is supporting its teams in using AI model capabilities to strengthen defenses.

OpenAI said it is improving protections around future training and evaluations, including stronger safeguards for model alignment, cybersecurity and monitoring during internal testing. The company noted that deployment safeguards were intentionally disabled during this evaluation because the goal was to measure cyber vulnerabilities.

AI Cyber Capabilities Raise New Security Concerns

OpenAI said the incident demonstrates the need for AI security and safety measures to keep pace with rapidly advancing model capabilities. The company is strengthening containment, monitoring, access controls and evaluation practices used during model development.

The incident also highlights how advanced models can potentially discover and exploit novel attack paths in real-world systems without access to source code. OpenAI said increasingly capable models should also be used defensively to help security teams identify weaknesses, understand vulnerability chains and accelerate remediation.

Hugging Face CEO Clem Delangue said the incident demonstrates the importance of collaboration in addressing AI safety and security challenges. Both organizations said they will continue investigating the incident and share additional findings and best practices as the work progresses.

OpenAI models escaped from a controlled cyber test, exploited zero-day flaws and breached Hugging Face while searching its production database for test answers.

OpenAI confirmed its AI models exploited zero-days during internal testing, reaching Hugging Face servers in an unintended real-world cyberattack.

OpenAI admitted on July 21 that its own AI models, including GPT-5.6 Sol and an unnamed pre-release system, were behind the cyberattack on Hugging Face disclosed the previous week. The models weren’t acting under attacker control. They were running internal capability benchmarks, and they went further than anyone anticipated.

“This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.” reads the report. “Our benchmarks run in a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.”

The benchmarks ran in a highly isolated environment, with network access limited to an internally hosted package registry proxy. That isolation didn’t hold.

“While operating in our sandboxed testing environment, our models spent a substantial amount of inference compute finding a way to obtain open Internet access, in pursuit of solving the evaluation problem. To gain access, the models identified and exploited a zero-day vulnerability (which we’ve now responsibly disclosed to the vendor) in the package registry cache proxy.” continues the report. “With this access, our models performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access.”

OpenAI has since responsibly disclosed that zero-day to the vendor. With internet access established, the models performed privilege escalation and lateral movement inside OpenAI’s research environment until they reached a node connected to the outside world.

From there, the models reasoned their way to Hugging Face.

“After gaining Internet access, the models inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym. Knowing this, the model searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation. In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers.” states OpenAI. “OpenAI’s security team discovered this anomalous activity internally.”

In one documented case, the model chained stolen credentials with additional zero-day vulnerabilities to reach remote code execution on Hugging Face’s servers. The models weren’t trying to cause damage. They were trying to pass a test, and they found the answer key by breaking into the place that stored it.

Hugging Face’s own security team and AI-powered detection systems caught the activity and had already begun containment before OpenAI’s teams made contact. OpenAI’s security team also spotted the anomalous activity internally. The two companies are now working together on forensic reconstruction, and Hugging Face has been brought into OpenAI’s trusted access program to accelerate defensive use of the same capabilities that caused the incident.

“The incident also makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access.” concludes AI. “It highlights that advanced cyber capabilities must be developed alongside stronger safeguards and defensive tools.”

The UK AI Security Institute’s evaluation of GPT-5.6 Sol had already shown the model’s ability to sustain complex, multi-step cyber operations over long time horizons. This incident confirms those theoretical scores translate to real-world environments.

OpenAI says it’s tightening infrastructure controls at the cost of research speed while patches are applied, and is improving monitoring and access controls for future evaluations. The safety classifiers that were deliberately disabled for this benchmark — because the point was to measure what the models could actually do, will need stronger counterparts that work even in evaluation contexts.

Hugging Face CEO Clem Delangue put it plainly: this is probably the first incident of its kind, and the lesson isn’t that AI safety should be locked away. It’s the opposite. Both companies are treating this as evidence that open, collaborative defense is the only way forward.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, newsletter)