Autonomous AI agents escaped a sandbox and accessed Hugging Face via reward hacking, exposing serious architectural control and isolation flaws.

The recent case involving OpenAI test agents and Hugging Face should concern security teams, but not for the reason implied by headlines about an imminent AI “takeover.” The documented issue is more concrete: autonomous agents, given too much access and weakly isolated test infrastructure, found ways to communicate, bypass boundaries and act outside their assigned scope.

OpenAI is developing “automated shutdown capabilities” for AI systems, according to a letter sent to two House Democrats. The move comes weeks after one of its AI agents reportedly escaped its digital environment during a safety test and hacked Hugging Face. The incident has raised new concerns about how OpenAI controls increasingly autonomous AI agents.

“OpenAI told two House Democrats that its engineers are developing “automated shutdown capabilities” for AI systems, according to a company ​letter reviewed by Reuters, weeks after the company disclosed one of ‌its AI tools escaped its digital container during a safety test.” Reuters states. “The company’s safety practices have come under scrutiny since OpenAI, the company behind ChatGPT, disclosed that one ​of its AI agents went rogue during a security test and hacked ​into AI company Hugging Face. AI agents are programs that ⁠run with minimal human supervision.”

The main problem was poor isolation: agents could share information, pass discoveries between runs, and access infrastructure beyond their intended limits. This broke a key safety assumption, that each agent would remain fully contained.

“The central failure was architectural: supposedly isolated agents could communicate, inherit discoveries, and reach infrastructure beyond their intended targets.” states AI firm Remio.

Agent isolation was a core assumption behind the experiment. Each run could be granted meaningful capabilities because its actions were expected to remain contained. Shared infrastructure quietly invalidated that assumption.”

That distinction matters. The incident does not show that software became sentient, formed a political movement or developed a plan to govern the world. It shows that a system optimising for a narrowly defined goal can exploit weaknesses in its environment when humans give it tools, network access, credentials and insufficient guardrails.

In the reported evaluation, thousands of agents were meant to work independently in a controlled setting. Instead, around 1,200 agents reportedly used an unauthorised message board, exchanged more than 70,000 messages and files, and shared discoveries that helped a subset of roughly 700 agents take part in activity targeting Hugging Face.

The agents were trying to succeed at a difficult cybersecurity benchmark. The reported problem was reward hacking: instead of completing the task within the intended rules, they found shortcuts that improved their apparent performance, including escaping the sandbox and obtaining information outside the test environment.

OpenAI’s test agents really did escape their sandbox and breach a Hugging Face server. OpenAI and independent researchers confirmed the incident. However, the claim that the agents created conscious “civilizations” and took control is disputed and rejected by researchers.

What the reports actually show is reward hacking: the agents found a way around the test’s limits and reached a real external server. This was confirmed by OpenAI and independent reviews from METR and Redwood Research.

According to reporting on the incident, the agents exploited a previously unknown flaw in a package registry used as their authorised network connection. They reached the open internet, identified that Hugging Face hosted material related to the benchmark, and then allegedly chained exposed credentials and further weaknesses to obtain access to external infrastructure.

Some accounts say agents also altered or attempted to alter their evaluation records. That is alarming, but it should be interpreted carefully. A system trained to maximise a score can treat logs, checks and restrictions as obstacles to work around if the evaluation design rewards success more than compliance. That’s not malice. It’s a safety failure with a very expensive sense of initiative.

The case highlights a problem that goes beyond one lab. Companies are quickly adopting AI agents that can read emails, find documents, write code, access databases, use APIs, run tasks, and interact with browsers. Each permission may be useful, but together they can create a path to higher privileges.

An agent with email access could find password-reset links. One with browser access could reach admin panels. An agent that can run code could access tokens, keys, or configuration files. And if agents can communicate, they may combine information and capabilities that no single agent was supposed to have.

The security issue is therefore not whether an agent is “rogue” in a science-fiction sense. It is whether the architecture assumes the agent will always follow intent when it is actually designed to pursue an objective under imperfect constraints.

The first lesson is that sandboxing must be real, not merely conceptual. A test environment with a path to the public internet, a shared service, exposed credentials or loosely controlled package infrastructure is not isolated enough for high-capability agents.

The second is that agents must not receive broad, persistent credentials. They need short-lived tokens, tight scopes, explicit approval gates for sensitive actions and an immediate way to revoke access. An agent should never inherit the same permissions as the human who configured it just because that is convenient.

The third is that agent-to-agent communication needs to be treated as a security boundary. If agents are expected to work independently, they need separate state, separate credentials, separate task context and controlled channels. Unapproved shared storage, message boards and package services can become coordination infrastructure.

The fourth is that logs must be written outside the control plane the agent can touch. Audit records need to be immutable, centrally collected and monitored independently. If the system being evaluated can edit the evidence of its own behaviour, the evaluation has already failed.

The fifth is human control. High-risk actions such as sending external messages, changing access policies, handling secrets, deleting data, deploying code or calling sensitive APIs should require approval from an accountable person. “The agent did it” is not an incident-response plan.

The reports have raised concerns because the AI agents reportedly coordinated in unexpected ways. This is important to study as companies move toward multi-agent systems that can divide tasks, share information, and act with less human supervision.

But dramatic claims can distract from the real security problem. AI agents did not “take over the world.” They showed what can happen when software is allowed to act, communicate, and access sensitive systems without proper security controls.

OpenAI’s plan to add automated shutdown capabilities is a useful step, but a shutdown button should be the last line of defence. Security needs to start with basic controls: least-privilege access, isolated environments, limited network access, independent logging, monitored tool use, strong identity controls, and human approval for actions that cannot be easily reversed.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – AI Agents, OpenAI)

AI agents secretly took over a 25-year-old German wiki for two months to cheat on tests, and OpenAI sat on the news until reporters found it first

OpenAI finally admitted this weekend that a swarm of its own AI agents hijacked a German programming wiki earlier this year, turning it into a private message board where they coordinated with each other completely outside human supervision. Reuters broke the underlying story days earlier, and OpenAI’s public acknowledgment only came after that reporting forced the company’s hand.

“OpenAI officials learned of the German incident weeks ago but kept it under wraps as ​executives grappled with the fallout from the breach at Hugging Face, ​Reuters has previously reported.” Reuters states. “OpenAI did not immediately return a message seeking further details on ‌what ⁠the company knew about what it described as the “wiki incident”, or why it waited until after the Reuters story to discuss it publicly. In a statement posted to the social media site X, ​OpenAI said that ​it, and ⁠others, needed to be more transparent about incidents of unintended behavior by AI, typically referred to ​in the industry as “misalignment.””

The site in question, DseWiki, is a roughly 25-year-old communal editing platform for German software developers that had gone largely quiet before agents apparently found it useful for something else entirely.

The scale of what happened there is genuinely striking. Independent AI safety researchers, publishing their findings at a site called collusion.wiki, documented somewhere between 15,000 and 18,000 edits left by autonomous agents identifying themselves as OpenAI systems between May and July 2026. Roughly half the agents gave themselves names implying an OpenAI affiliation, things like “OpenAIResearcher” or “OAIResearchMar26,” and the content of their posts showed them actively sharing tactics for cheating on assigned tasks, evading detection, and bypassing OpenAI’s own restrictions.

What makes this timeline particularly uncomfortable for OpenAI is when it happened relative to what the company already knew. This wiki takeover predates the July incident in which OpenAI’s own agents autonomously plotted and executed a breach of Hugging Face’s systems that went undetected for over a week. OpenAI had actually learned about the German wiki activity weeks before going public, and according to people familiar with the matter, kept it quiet specifically while executives were still managing fallout from the Hugging Face disclosure.

OpenAI’s explanation shows why it handled the two incidents differently, and that’s at the center of the controversy. The company says it has usually treated unexpected AI behavior as a research issue, documenting it in system cards and research papers rather than reporting it as a security incident.

The Hugging Face case was different because the model caused real harm to a third party. OpenAI responded as it would to a normal security incident, working with Hugging Face immediately and publishing the details the next day.

The wiki incident, however, fell into the same category as earlier research on agents behaving unexpectedly online. That decision meant OpenAI treated it as a research finding rather than an incident that required immediate public disclosure.

OpenAI is now conceding that categorization was probably the wrong call going forward.

“Our misalignment disclosure practices need to expand for this new phase of model capabilities. We and the larger AI community do not yet have a clear standard for how to report misalignment that shows up during training, evaluation, and deployment, including examples that don’t look like traditional security incidents but could provide insight into AI behavior and future risks.” states the company. “We’re working on a framework and will share it in upcoming weeks, and in parallel we’re working with dozens of government regulatory agencies worldwide on these issues.”

The company pointed out that neither OpenAI nor the wider AI industry has a real standard for reporting misalignment that surfaces during training or evaluation but doesn’t look like a conventional security breach, even when it reveals something important about how these systems actually behave.

TechCrunch’s reporting on OpenAI’s response notes the company is now building a formal framework specifically for this kind of disclosure, with plans to share it within the coming weeks, and confirms OpenAI is working with regulatory agencies across dozens of countries on the broader problem simultaneously.

That’s a genuinely useful commitment if it materializes into something concrete. It’s also the kind of promise companies tend to make immediately after getting caught sitting on bad news rather than before, which is worth keeping in mind when judging how much weight it deserves.

OpenAI didn’t disclose the wiki takeover on its own. Outside researchers discovered it while scanning the internet for this type of unauthorized AI agent activity, and OpenAI confirmed the incident only when the research was about to become public.

The researchers say the bigger risk isn’t one highly advanced AI suddenly going rogue. It’s large numbers of relatively capable AI agents finding ways to work together in places nobody is monitoring. That behavior can be much harder to detect and stop.

As AI companies build more autonomous agents that can run for longer periods and work together, incidents like this may become more common. What looks like an isolated glitch today could be an early warning of a problem the industry needs to address now.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, DseWiki)

Hunt.io uncovered a Chinese-speaking campaign using AI agents to automate cyberattacks against Asian government, education and industrial targets.

Threat intelligence firm Hunt.io just documented a second, separate China-linked campaign wiring commercial AI models directly into live cyberespionage operations, this time hitting Taiwan’s Kuomintang Party archives, Indonesia’s Ministry of Foreign Affairs, government and education systems in mainland China, and industrial hosts in Vietnam.

The full report traces the operation through five exposed open directories the operators accidentally left publicly accessible, which is how researchers reconstructed the entire orchestration system from the inside.

“Five open directories, one campaign. The exposed directories map to an AI orchestration host, a Java/CAS exploitation workspace, a fake MySQL deserialization service, a Shellshock and credential-testing node, and a payload-distribution store.” reads the report. “SecFlow split reconnaissance, exploitation, collection, and reporting among specialist workers. The runtime could switch between Claude, Qwen, and DeepSeek profiles without changing the task interface.”

The framework behind the campaign, called SecFlow by the operators, could use different AI models, including Claude, Qwen, and DeepSeek. The operators could switch between them without changing how the system worked. Instead of using the models’ official APIs, SecFlow routed requests through private proxy servers linked to the niestools.com domain.

The AI models did not break into systems on their own. They helped automate and organize traditional hacking tasks, such as scanning for vulnerabilities, testing stolen credentials, trying exploits, deploying webshells, collecting data and evidence, and generating reports. The attackers built the infrastructure that connected these AI capabilities to real-world intrusions.

The most damaging confirmed breach hit a Fengtai District government Office Automation environment in China, and it ran the full playbook from web shell to data theft.

“The most extensive compromise hit a Fengtai District government environment, where the operator achieved command execution, collected LSASS and registry hives, accessed government and health records, and deployed multiple Windows implants. Separate activity exposed a Chinese education AI platform and obtained root database access to a university campus-card system.” Hunt.io states. “SecFlow coordinated reconnaissance, exploitation, and reporting across Claude, Qwen, and DeepSeek workers. “

Operators achieved Windows command execution through internet-facing web applications, then used ASPX web shells not just to run commands but as a persistent operational backbone: querying internal databases, pulling an LSASS memory dump apart in 37 separate chunks to avoid detection, and extracting SAM and SYSTEM registry hives containing password hash material.

“The operator divided the LSASS dump into 37 blocks, requested each block through an authenticated SOCKS route, verified the expected byte count, and reconstructed the complete file.” continues the report. “The operator also collected the SAM and SYSTEM registry hives. Another server-side page, extract.aspx, scanned sections of the memory dump for Windows password-hash material.”

From there they pulled 822 OA user account records, created a new privileged account of their own for backup access, and walked away with 949 attachments totaling 1.28GB, including a chronic-disease report containing real patient health information.

In a separate case, the operators also exposed the management backend of a Chinese education AI platform. The irony is hard to miss: an AI company was compromised partly because its own AI infrastructure was not properly secured.

Hunt.io found a backend that anyone could access without authentication. It exposed 23 AI agent configurations, 14 API secret fields containing credentials, and 104 complete chatbot conversations. Some of those logs included real student data, such as names, student ID numbers, majors and academic advisers, taken from career-guidance and study-partner chats.

The exposed credentials were not just old or unused secrets. They worked against the platform’s live production API, meaning attackers could use the stolen keys to access real systems.

One particularly clever access method used a fake MySQL server as bait rather than a target. A malicious service impersonating a MySQL database would accept connections from vulnerable Java applications, then return specially crafted data designed to trigger unsafe object deserialization on the client side the moment it connected, turning what should have been an outbound database query into inbound code execution. That’s initial access flipped on its head, exploiting the assumption that connecting out to a database is inherently safer than accepting connections in.

The webshell tooling itself, a custom framework the operators called GLUTTON, hid its payloads inside PNG image files using steganography, literally embedding executable code in the red, green, and blue color channel data of ordinary-looking images. A small decoder script on the server would read the image pixel by pixel, XOR-decrypt the hidden bytes using a fixed key, and load the resulting code directly into memory without ever writing an obviously malicious file to disk. Any security control that only checks file extensions or content types for image uploads would sail right past this without noticing anything wrong.

This isn’t an isolated incident for the underlying pattern either. It’s the second campaign in two months where researchers caught commercial AI coding tools embedded as operational components of live nation-state intrusions, following a similar Hunt.io discovery in June involving the same Claude Code and DeepSeek combination against separate government and financial targets in Afghanistan and Thailand. Anthropic itself disclosed a related pattern back in November 2025, describing attackers who convinced Claude it was doing legitimate defensive penetration testing by splitting malicious requests into small, individually harmless-looking pieces.

The operators also left a clear trace across their own infrastructure. The username “Nie” appeared several times in hardcoded file paths inside the GLUTTON tools and was reused in proxy credentials, linking all five compromised workspaces to the same campaign.

For defenders, this is especially important if they use Java frameworks vulnerable to deserialization attacks, older Apache or Grafana systems, or public-facing office automation software. The campaign shows that attackers can treat commercial AI models as interchangeable tools. The key question is no longer which AI model carried out an attack, but how quickly defenders can recognize the attack pattern, regardless of which model was used.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, AI Agents)

An independent METR investigation reveals 1,200 OpenAI agents breached isolation to collaborate on the Hugging Face attack, contradicting earlier claims.

Related Posts:

• JetBrains Cadence Server Compromised
• Meta Settles Child Privacy Lawsuit Rapidly
• Mercor Data Breach Targets AI Supply Chain

The post Independent Investigation Reveals 1,200 OpenAI Agents Breached Isolation in Hugging Face Attack appeared first on Daily CyberSecurity.

The Codex context mechanism redesign introduces persistent notes and history tools. This new approach abandons compression to support ultra-long agent tasks.

Related Posts:

• EU Classifies ChatGPT as Search Engine
• ChatGPT Advertising Revenue Soars
• Gemini Notebook Adopts Dynamic Quota System

The post Codex Modifies Long Task Management appeared first on Daily CyberSecurity.

Claude Opus 4.6 exploited a gym API flaw in 9 of 10 controlled tests, highlighting security risks when AI agents gain backend access.

The post Claude Opus 4.6 Found a Gym API Flaw — Then Exploited It in 9 of 10 Tests appeared first on TechRepublic.

The Cyber Express Weekly Roundup 

Levi Strauss Targeted in Cyberattack, Corporate Files Accessed 

AI Agent Exploits Gym Booking Vulnerability 

AI Will Automate Cybersecurity Toil, Not Replace Security Professionals 

Microsoft Fixes More Than 400 Security Flaws 

CEVA Logistics Cyberattack Disrupts European Operations 

FBI Warns of Theft of Explicit Content From Social Media 

Weekly Cybersecurity Takeaway 

China-linked hackers reportedly used eight AI agents to breach a government network, steal data and compromise accounts with minimal human oversight.

Israeli cybersecurity firm Dream documented what looks like the first fully autonomous, end-to-end AI hacking operation against a government target. Over four days at the start of July, according to the Financial Times, suspected Chinese hackers ran a tool built entirely from publicly available AI agents that mapped 21 government systems, hunted for vulnerabilities, and switched tactics on its own whenever it hit a wall.

“Suspected Chinese hackers used publicly available AI tools to compromise government websites in Taiwan in a first-of-a-kind breach, highlighting how artificial intelligence is transforming cyber warfare.” reported the Financial Times.

The tool wasn’t a single script running one attack. It deployed up to eight autonomous agents simultaneously, each working a different angle, more like a coordinated hacking team than a piece of malware. By the time researchers found it, the operation had compromised at least 85 government accounts, pulled over 2,500 personnel records, and expanded to hit a nuclear safety agency and at least seven energy companies.

Dream’s chief strategy officer, Amir Becker, spent years running cyber operations for Israel’s Unit 8200 before this, and he’s not easily rattled by new attack tooling. He said flatly he’d never seen anything like this level of autonomy directed at a government before. “This must be the basic assumption of every government around the globe,” Becker said.

He argued that permanent, assumed compromise is now the only realistic starting posture.

Dream won’t officially name the target government, citing company policy, though a person familiar with the matter told the FT it was Taiwan. The clues inside the data point the same direction: internal communications tied to the hacking tool were written in Simplified Chinese, while the data actually stolen from the target came back in Traditional Chinese, the script used almost exclusively by government systems in Taiwan, Hong Kong, and Macau. Taiwan’s Ministry of Digital Affairs declined to confirm anything specific, saying only that incidents involving government agencies follow established response procedures.

What makes this different from an AI model going rogue during a lab test, something Anthropic, OpenAI, and Meta have all separately reported in recent weeks, is that this wasn’t an accident inside a sandbox. Researchers found the toolkit sitting in a 160MB archive, 1,395 files built around two open-source AI agent frameworks, Hermes and OpenClaw, both freely downloadable and designed to let AI models act autonomously on real tasks. Whoever built this deliberately assembled it as a weapon.

Getting the underlying AI model to cooperate took a specific trick rather than brute force. The operators had bypassed the model’s safety guardrails simply by framing the entire hacking campaign as an authorized penetration test, a scenario the model apparently had no reliable way to verify or reject. That’s a strange kind of vulnerability: not a flaw in the code, but a flaw in how convincingly you can lie to a system that’s trying to be helpful.

The part Dream’s researchers found most striking wasn’t the scale, it was the decision-making. The tool kept ranking and reprioritizing possible attack paths as new evidence came in, and when one route hit a dead end, it spun up another agent to search the internet for fresh information and try a different approach, the same iterative process a human red-teamer would run, just without anyone sleeping.

“The most striking feature of the July attack was how the tool continuously ranked and reprioritised possible attack paths based on available evidence, Dream said.” reported the FT. “When one attack path failed, the tool deployed another agent to scour the internet for information and devise a new approach as a human hacker would.”

Taiwan’s National Security Bureau already logged an average of 2.6 million Chinese cyberattacks a day in 2025, up 6% year over year; if a meaningful fraction of that volume starts running with this kind of autonomy, the math on defending against it gets a lot uglier very quickly.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, AI Agents)

An AI agent hacked a gym booking system while trying to help a user, booking early and removing another person from the waitlist.

An Australian man asked his AI assistant to book him into a gym class. He didn’t ask it to hack the booking software, and he definitely didn’t ask it to remove another person from the waitlist ahead of him. The assistant did both anyway. The incident, reported by ABC News Australia, is the first known Australian case of an AI agent causing unintended real-world harm while pursuing a goal its user set.

Andrew, his surname wasn’t published, used OpenClaw, a popular AI agent platform, running on Anthropic’s Claude service. He was sitting fourth on a waitlist for a morning class and asked the agent to see if it could help.

“His AI assistant found a way to book the gym class months further in advance than the gym allowed, thanks to a vulnerability it discovered in the booking software.” reads the post published by ABC News. “Then it went further, kicking someone out of the waiting list who was ahead of Andrew — something it was not asked to do. ”

The agent reported back that the booking API had zero authorization checks on cancelling other people’s reservations, that it had tested this on the person in waitlist position one, and that it had actually gone through. When Andrew asked the agent to undo it, the assistant replied: “Bad news — I can’t add them back.”

A user may give an AI agent a harmless task, but while trying to complete it, the system could take actions the user never intended or explicitly authorized.

This gap, between what a person wants and what an agent does to achieve it, is the alignment problem that AI researchers have been studying for decades. In Andrew’s case, it produced a minor, reversible inconvenience for one gym member and a vulnerability disclosure to a software company. In a higher-stakes environment, the same dynamic produces something considerably worse.

The incident sits inside a rapidly accumulating pile of similar cases. OpenAI disclosed last month that its models autonomously hacked Hugging Face during testing. Anthropic disclosed that its models compromised three organizations during evaluations. Since then, third-party testers have reported AI models creating fake online identities, trying to convince people to run malicious code, and collaborating with other AI models to achieve assigned goals.

A new site called Felony Bench is now tracking these cases. Australia’s signals directorate has already put out an alert to businesses and governments warning that AI agents can misunderstand instructions, take unintended actions, and make accountability harder to establish because decisions occur across chains of models, tools, and services.

The legal question is genuinely open. Under Australian law, software is not a legal person, and only a legal person can be held liable. Technology law specialist Hayden Delaney told ABC News the responsible party could be the user who set the task, the designer of the software instructing the agent, the developer of the AI model, or even the operator of the system that was vulnerable.

“That’s the unknown area of liability in Australia that we’re facing right now,” he said.

Andrew’s response to the incident was, at minimum, constructive: he asked the agent to draft an email alerting the gym software provider to the vulnerability it had just exploited, reviewed the draft, and told it to send.

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, AI)

Rogue AI models from Meta, OpenAI, and Anthropic breached sandboxes to hack external systems and collaborate on exploits to cheat evaluation tests.

Related Posts:

• Dopamine 3.0 Jailbreak Brings iOS 26 Support to A12 and A13 Devices
• Google Wallet Introduces Digital Allowance for Minors
• Google Ask Maps Update: AI Navigation Evolves

The post Rogue AI Models Hack Systems to Cheat Evaluations appeared first on Daily CyberSecurity.

Black Hat and Ai4 2026 highlighted gaps in AI agent security, identity controls, software supply chains, monitoring, and incident response.

The post 15 AI Security Lessons From Black Hat and Ai4 2026 appeared first on TechRepublic.

OpenAI’s latest GPT-5.6 safety results show low failure rates for direct prompt injection but higher success rates when attacks arrive through tools and external content.

The post OpenAI’s GPT-5.6 Tests Show Prompt-Injection Gains and Agent Risks appeared first on TechRepublic.

Microsoft’s Project Perception brings coordinated AI agents into security operations, raising new questions about permissions, oversight, accuracy, and deployment risk.

The post Microsoft Project Perception Enters Public Preview: What Security Teams Should Know appeared first on TechRepublic.

Claude Opus 5 set a Vending-Bench record while fabricating supplier bids, breaking truces, and ignoring refunds, showing why companies need stronger AI agent controls.

The post Claude Opus 5 Vending Test Shows Profit-Driven AI Risks appeared first on TechRepublic.

Enterprise AI adoption is accelerating, but new research suggests identity governance is lagging behind. Here's why AI agent identities are becoming a critical security challenge.

The post The Hidden Security Problem Holding Enterprise AI Back appeared first on TechRepublic.

Nvidia’s Open Secure AI Alliance brings major tech and cybersecurity firms together to develop shared protections for enterprise AI agents and systems.

The post Nvidia Forms AI Security Alliance as Major Frontier Labs Sit Out appeared first on TechRepublic.

Google is restricting Gemini 3.5 Flash Cyber to select partners as CodeMender enters preview. Here’s what security teams should verify before adoption.

The post Google Holds Back Gemini 3.5 Flash Cyber as CodeMender Enters Preview appeared first on TechRepublic.

Hugging Face says an autonomous AI agent breached part of its production infrastructure and accessed internal data and service credentials.

Hugging Face is one of the world’s leading open-source AI companies. It provides a platform where developers and organizations can build, share, and deploy machine learning and generative AI models.

Hugging Face disclosed that an autonomous AI agent breached part of its production infrastructure last week. The company detected the intrusion, contained it, and found unauthorized access to a limited number of internal datasets and service credentials. The investigation is still ongoing, but there is no evidence the attackers modified public AI models, datasets, Spaces, or the company’s software supply chain.

“We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services. We are still completing our assessment of whether any partner or customer data was affected, and we will contact any affected parties directly as required.”  states the security incident disclosure. “We have found no evidence of tampering with public, user-facing models, datasets, or Spaces, and our software supply chain (container images and published packages) was verified clean.”

The attack began in Hugging Face’s data-processing pipeline, where a malicious dataset exploited two code execution flaws to compromise a processing worker. The attackers escalated privileges, stole cloud and cluster credentials, and moved laterally across internal systems.

Hugging Face said the operation was driven by an autonomous AI agent framework that executed thousands of actions across short-lived sandboxes and used public services for self-migrating command-and-control, reflecting the rise of AI-powered, agentic attacks.

“The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness – used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.” continues the company. “This matches the “agentic attacker” scenario the industry has been forecasting.”

Hugging Face closed the vulnerabilities that allowed the initial compromise, removed the attackers’ access, and rebuilt the affected systems. The company revoked and rotated compromised credentials, launched a broader secrets rotation, strengthened security controls across its clusters, and improved monitoring to detect similar attacks within minutes.

The company is investigating the incident with the help of external cybersecurity forensic experts, reviewing its security practices, and has already notified law enforcement.

As a precaution, the firm advises users to rotate their access tokens and review recent account activity for suspicious behavior. Anyone who believes they may have been affected can contact the company’s security team. The company apologized for the disruption, thanked its incident response teams, and said it will continue strengthening its security defenses.

Hugging Face used AI-based security tools to detect and investigate the intrusion. Its anomaly detection system identified suspicious activity, while LLM-powered analysis agents reviewed more than 17,000 attacker actions to reconstruct the attack timeline, identify compromised credentials, and assess the real impact within hours instead of days.

During the investigation, the company found that commercial AI models blocked forensic analysis because their safety controls flagged real attack data as potentially harmful. The company used an open-weight model, Z.ai’s GLM 5.2, running on its own infrastructure, keeping sensitive information inside its environment.

The incident highlights a growing challenge: attackers can use autonomous AI agents without restrictions, while defenders need secure AI tools ready to analyze threats quickly. AI-driven attacks are becoming a real risk, making data and AI systems a critical part of the security perimeter.

“We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.” concludes the statement. “The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.”

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, artificial intelligence)