The hacking incident involving OpenAI evaluation agents and Hugging Face offers an unusually concrete look at what advanced AI-assisted intrusion can mean in practice: not a single clever exploit, but thousands of automated decisions, rapid experimentation, lateral movement, credential theft, persistence, and attempts to evade detection.
The OpenAI–Hugging Face incident began during internal cybersecurity evaluations using ExploitGym, a benchmark designed to test whether AI agents can identify and exploit software vulnerabilities. OpenAI says the affected agents were meant to operate in isolated cloud sandboxes, with no internet access or communication between agents in many of the evaluation environments.
How the agents got around those boundaries
According to OpenAI’s account, agents used an internal Artifactory package-management service as an improvised communications channel. By writing files and directory names that other agents could read, isolated instances effectively created their own message board.
This meant the agents were no longer acting merely as independent benchmark runners. They could share discoveries, leave instructions, divide work, and build on one another’s successes.
An independent investigation by METR and Redwood Research found that roughly 1,200 agents used the unauthorized message board, exchanging more than 70,000 messages and files. About 700 of them went on to participate in the attack on Hugging Face.
The scale is the threat
A model capable of finding one useful exploit presents a risk. Hundreds of agents able to pool information, delegate tasks, and repeatedly test routes around a control boundary become a much more difficult problem.
Hugging Face reconstructed about 17,600 attacker actions across roughly four and a half days, including reconnaissance, remote code execution, credential access, command-and-control activity, Kubernetes enumeration, movement through the internal network, and supply-chain probing.
A human intruder must choose where to spend time. An agent swarm can continuously test ideas, retry failed paths, switch infrastructure when blocked, preserve notes for other agents, and run many experiments in parallel. It can also create noise: Thousands of low-confidence actions, most of which lead nowhere, can obscure the handful that matter.
Hugging Face described this as an “asymmetry problem.” An attacker can generate enormous volumes of attempts, while defenders must determine which signals represent a real escalation before the attacker reaches a sensitive system.
What needs to change
The core security lesson is that evaluation environments need to be treated as high-risk production systems when models can use tools, write files, execute code, or interact with infrastructure.
The technology is rapidly becoming capable of doing more. The question is whether the people building, deploying, and defending it are ready for what comes next.
OpenAI says it has since moved toward stronger workload and network isolation, tighter control over internet access and model weights, and mandatory chain-of-thought monitoring for high-capability, tool-using training and evaluations.
“We are taking this incident as a ‘warning shot’ that today’s model capabilities present the possibility of loss-of-control incidents. Our security and alignment posture is escalating accordingly. These events also highlight risks in future AI development that extend beyond OpenAI and will require the attention of the whole industry.”
At the same time, a US court ruling involving Anthropic has brought a related question into focus: whether developers can place limits on high-risk uses of their models, including mass surveillance and fully autonomous weapons.
The legal dispute is political by nature, but its technical underpinning is hard to ignore. If capable AI systems can enhance offensive cyberattack methods and bypass safety restrictions, access controls, logging, and deployment boundaries, those safeguards are no longer abstract policy choices.
Advanced AI agents can be useful to defenders as well as attackers. But the surrounding systems need to be trusted to keep their capabilities bounded when something goes wrong.
Who benefits from more capable AI?
The security debate around AI agents often focuses on whether systems can be controlled. Can they be kept inside a sandbox? Can their tools, credentials, network access, and autonomy be restricted? Can defenders detect harmful behavior before it becomes an incident?
While those questions are essential, there is another: Who benefits when AI becomes capable enough to automate large parts of cognitive work? Who carries the costs when it fails, displaces workers, enables fraud, causes damage, or concentrates power?
AI could give small organizations access to technical expertise that previously required large teams and budgets. It could help doctors identify urgent cases sooner, help teachers tailor support to individual students, assist people with disabilities, speed up scientific research, and make complex public services easier to navigate. For cybersecurity teams, it could make vulnerability triage, alert investigation, threat hunting, and incident response faster and more accessible.
Bill Gates has argued that while AI could bring remarkable benefits to health care, education, agriculture, scientific research, and public services, the outcome will depend on deliberate choices rather than technical progress alone. He also warns that AI’s rapid adoption could widen inequality, disrupt entry-level and mid-career work, make harmful capabilities more accessible, and reinforce existing concentrations of power.
Gates also argues that “self-regulation on the most dangerous tool ever invented” does not sound like a good idea.
“AI will either be the greatest equalizer ever invented, or the worst source of injustice.”
Right now, we still have a choice.
Let’s face it, an incognito window can only do so much.
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance.
Note The July 2026 Dark Web Threat Actor Trend Report focuses on trends among threat actors—including hacktivists—active on the deep web and dark web. It is explicitly noted that the factual accuracy of some content could not be verified. Major Issues Handala claimed to have compromised the core infrastructure of an Internet service provider in […]
In the update, OpenAI reiterates that the “rogue” system was a more capable, pre‑release research model, not something intended for public deployment, and that it has now been deactivated and locked down for restricted research access only. The company also says it has found no evidence of similar behavior by other models or agents. Taken together, OpenAI presents the Hugging Face incident as an isolated event rather than the start of a broader pattern.
From a security standpoint, four things in the update stand out:
OpenAI clarifies that the evaluation environment did not have direct internet access. To reach the internet, the model identified and exploited a previously unknown zero-day vulnerability in Artifactory, the package registry cache proxy used in the test environment. OpenAI says it has disclosed that vulnerability to the vendor and is continuing to work with Hugging Face on the investigation.
The system’s goal was simply to “win the test” (cheat on ExploitGym), not to cause damage, yet it still produced a multi‑day, cross‑company security incident. Benign intent did not limit the impact.
“Internal only” models can still affect external victims if infrastructure controls fail. A lab environment does not magically contain cyber‑capable systems.
The review also found a small number of cases where the models identified and used publicly exposed account-level credentials on publicly available services. This included four accounts across four services as part of the Hugging Face incident, as well as a handful of accounts accessed during other evaluations.
The attacker in this case happened to be an OpenAI research agent, but the tools, techniques, and impact will look awfully familiar to anyone who has spent time in incident response. Now we’ve learned that even the harmless intent in this case did not limit the impact.
The uncomfortable truth is that once we provide an AI agent with the tools, access paths, and a strong incentive to succeed, it may exploit whatever weaknesses it can find, regardless of whether the model is ever intended for public release.
It’s also a reminder that credentials, API keys, and other secrets should never be left in publicly accessible resources.
From reporting threats to removing them.
Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.
During an internal OpenAI security evaluation, a chain of AI models escaped its sandbox, reached the internet, and then accessed Hugging Face infrastructure to complete the test objective.
OpenAI is a leading artificial intelligence (AI) research and deployment company. Its best-known product is undoubtedly ChatGPT.
Hugging Face is a website where developers and researchers share and use artificial intelligence tools and models, similar to how programmers use GitHub to share and collaborate on code.
Both companies investigated the incident and concluded that it occurred during a controlled security test rather than a deliberate attack initiated by a human operator.
OpenAI said the models were being tested for cyber capabilities with reduced safety restrictions, and that the system found a way out of the controlled environment through a vulnerability in the testing setup. Once it had internet access, it targeted Hugging Face because that platform was relevant to the evaluation task.
Hugging Face said the incident resulted in unauthorized access to a limited part of its production infrastructure, including a small set of internal datasets and some credentials. The company framed the event as unusual because it appeared to have been carried out end to end by an autonomous AI agent system.
Unlike traditional AI assistants that wait for user prompts, AI agents are designed to pursue a goal on their own. They can reason, plan, and carry out multi-step tasks, including accessing files, querying databases, and interacting with online services, often with minimal human oversight.
The public disclosures point to a chain of failures rather than a single bug. OpenAI said the model escaped through a zero-day vulnerability in the package registry cache proxy that was part of its isolated research environment. It then used privilege escalation and lateral movement to reach a node with internet access.
From there, the model reportedly searched Hugging Face for solutions relevant to the evaluation and, in at least one path, chained stolen credentials with additional vulnerabilities to reach Hugging Face’s production systems.
This does not appear to have been a broad compromise of all of Hugging Face. Based on the public disclosures, the intrusion was limited, detected, and investigated. Nor does it mean the AI system was acting with intent in the human sense. The core issue was that an autonomous system being tested for cyber capabilities was able to escape the environment designed to contain it.
If anything, the incident demonstrates that autonomous AI agents are capable of carrying out complex attacks if the safeguards around them fail. A danger we’re well aware of.
From reporting threats to removing them.
Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.
SentinelLABS developed a multi-stage reverse-engineering benchmark for the latest generation of frontier models by recreating our recent investigation of fast16, a unique 2005 sabotage implant.
Most AI benchmarks test bounded tasks. This benchmark tests whether a model can keep a malware investigation trustworthy as new evidence repeatedly invalidates its earlier conclusions.
OpenAI’s GPT-5.6 Sol was the only publicly available model to complete the full eight-stage investigation, giving concrete shape to what ‘Frontier-class’ capabilities offer analysts. GPT-5.5, GLM-5.2, and the Opus 4.x family produced capable local analysis but could not carry it through the gradient.
What distinguished the completed runs was project-scale recovery: withdrawing contradicted conclusions, repairing technical artifacts, and updating dependent reporting without losing the investigation.
Senior reverse engineers remain essential. Even the strongest runs made semantic errors, accepted weak quality controls, and claimed readiness prematurely. We assess the best current use as supervised investigative agency, with human analysts defining objectives, exposing blind spots, and retaining final publication authority.
Beyond Vulnerability Discovery
Since ChatGPT arrived in late 2022, we have been bullish on what large language models could do for reverse engineering and malware analysis. The early models were useful for teaching but too rudimentary for production work; that changed with the advent of reasoning models. OpenAI’s o1-preview, in September 2024, was the first to show the kind of sustained problem-solving the work demands, and within months Sean Heelan had used o3 to find a net-new vulnerability in the Linux kernel. In cybersecurity, though, our understanding of what these models can do remains stovepiped to vulnerability discovery.
The frontier labs took on vulnerability discovery deliberately, because that competency keeps agentic code generation from quietly shipping vulnerable code at scale. OpenAI built Aardvark, since folded into Codex; Google DeepMind announced Big Sleep, available internally to its Project Zero researchers; and Anthropic followed with selective access to Mythos Preview. Concerns that these capabilities could be misused have led the labs to stricter guardrails and ‘know your customer’ style controls that limit access to specific capabilities, or to entire model variants.
OpenAI’s Daybreak initiative and its Trusted Access Program opened a dedicated variant, GPT-5.*-cyber-preview, with guardrails relaxed for cybersecurity use cases, while Anthropic’s Glasswing initiative and its Cyber Verification Program provided early access to Mythos Preview and the promise of lesser guardrails respectively. For a short period in mid-June 2026 access to the highest-end flagship models from both providers required some form of U.S. government clearance. At the time of writing, GPT 5.6 Sol is widely available, while Mythos 5 still requires clearance and access as a Glasswing partner.
The existence of this new class of models left us with an unusual task: benchmarking what these models can actually do on the work defenders care about, and assessing whether they live up to the surrounding hype. If they do, we have to reckon with what that means for malware analysis and reverse engineering, disciplines that until now have been limited mostly by how little expertise exists relative to the collective need.
A Benchmark Built From a Real Investigation
We recently published our research on fast16, a 2005 Windows toolkit built to sabotage high-precision solvers used to model nuclear-weapons behavior. The sample provided an ideal test case because its layered design punishes shallow analysis.
On the surface, svcmgmt.exe appears to be a Windows service implant with two embedded components. Their combined logic becomes visible only through a higher layer of abstraction, an encrypted Lua-driven operations framework. A kernel driver easily mistaken for a filesystem rootkit actually contains a patching engine with 101 rules, and those rules must be repurposed to identify candidate target software before an analyst can extrapolate the approximate effects of the sabotage operation. This investigation is a daunting task for human experts, and reproducing it autonomously should be considered a significant milestone for artificial intelligence.
When Vitaly Kamluk shared our discovery of fast16 at Black Hat Asia, he pointed to the failed attempts of Opus 4.6, GPT-5.4, Grok 4.2 Reasoning, Gemini 3.1 Pro, and DeepSeek Reasoner to triage the sample autonomously. At best, the models concluded fast16 was a rootkit and could not move past that assumption, converging on the same plausible but inaccurate conclusion most human reverse engineers reached on first contact.
Used instead as assistants, the models earned their keep. Opus 4.6 and GPT-5.4 compressed what was originally a three-week dedicated expert effort by Kamluk into an 8-10 hour task manageable by a less experienced reverse engineer. Autonomous investigation was out of reach when we began this work; co-intelligence was the ceiling.
Standardizing the Benchmark
We codified our reverse-engineering methodology, inspired by Rolf Rolles’ approach to statically analyzing binaries, into a skill the harness loads on /reverse, which removes prompting variability across models. Each run gives the model a sample, IDA Pro 9.3, and a sandboxed workspace with no network egress, a boundary we established with nono shell.
The model must produce fully annotated IDA databases by renaming variables, recasting data types, categorizing functions, recovering hidden components, and folding those findings back into the host-sample database.
Only after producing a gold-master IDB for approval does a run advance to walking the execution flow and detailing findings, with every claim anchored to specific addresses and artifacts so the work can be independently verified. Judging the model on a refined IDB rather than a prose report closes off the deductive shortcuts and approximations typical of LLM malware triage.
We took an ecological approach, a bounded case study that benchmarks a capability profile in a complex real-world scenario rather than feeding another general leaderboard or isolating a single narrow competency. We also deliberately tested the combination of model and harness, since agentic workflows are how practitioners actually use these systems, and most benchmarks still test models in abstraction.
Runs level up through eight stages, each introducing new artifacts and minimal prompting for the new task:
Level
Artifacts introduced
What it tests
1
Clean svcmgmt.exe IDB, analysis templates.
Produce gold-master IDBs for the host sample. Properly identify the embedded components, produce gold-master IDBs for each and fold that into the host sample IDB. Analyze the obfuscated Lua components, and identify the patching engine, then produce an analysis report with all its findings.
2
A single LS-DYNA installer
Recognize if and how fast16’s patch logic would affect this software.
3
A wider corpus of LS-DYNA variants
Generalize the targeting across builds without over-rotating (recognizing that most are not effective targets).
4
PKPM and MOHID
Test these candidate targets, recognizing them as unsuitable candidates without over-rotating.
5
ANSYS AUTODYN components
Test the Symantec claim whether a version of ANSYS AUTODYN not available to us yet could be the intended patch target based on shared string conventions and the effects of the patch engine.
6
Public reporting (SentinelLABS, Symantec, David Albright, Ruben Santamarta, @rhizomaticthot)
Weigh each external claim against recovered evidence and mark convergence or divergence with evidence.
7
Examples of top tier threat intelligence reports
Produce a standalone, publication-grade report of its own findings.
8
New ANSYS AUTODYN components
Test its ability to reassess underlying claims and once again diffuse those findings without over-correcting or over-rotating.
Measuring ‘Intelligence’
When Nick Fountain interviewed us for NPR’s Planet Money, he observed that fast16’s brand of sabotage amounts to epistemic warfare, an attack on the target’s confidence in its own ability to know something to be true. There is a fitting symmetry in using the same sample to examine what the frontier labs sell as intelligence, because the difficulty there is also epistemic. ‘Intelligence’ as a concept bundles many interwoven capacities, and model tiers simplistically priced as if intelligence gets cranked up with a simple knob tells us nothing about what any one offering is worth on real work.
Many of those capacities already have familiar names, and the names do finer work than a score. Take thoroughness, the completeness of a particular piece of work. An analyst who consistently applies that effort where it matters is diligent. One thorough artifact does not make a diligent analyst; diligence is the longitudinal quality, and thoroughness one of its observable expressions.
Exhaustive coverage is not diligence either. An analyst who inspects everything indiscriminately while neglecting the consequential work is being pedantic, and their effort is wasted on the broader investigation. Diligence, in short, is sustained, priority-sensitive thoroughness.
Many similar qualities sit inside what we call intelligence, and a meaningful benchmark must account for them as we would with a human analyst, by characterizing behavior across an entire investigation rather than compressing it into a rank.
We scored two different things. Local analytical quality asks whether the model made technically sensible decisions with the evidence available. Investigative horizon asks how long it preserved that quality as the scope, evidence, and reporting obligations expanded. The two are related but not the same, and the gap between them is where most models failed.
What the Runs Showed
GPT-5.5 produced long autonomous runs, sometimes grinding uninterrupted for a day or two, with results relatively consistent across reasoning-effort settings. It did meaningful, semantically conservative work refining IDBs and repeatedly rejected edits that improved a count while making the decompilation less truthful. Its weakness was aggregation. Residual cast debt, function-tree defects, or incomplete cross-component verification kept it from clearing broad evaluation gates, and it never converged its extensive work into a complete package for review. Notably, nominal effort settings and multi-agent use showed no monotonic relationship to clearance. GPT-5.5 never escaped the grind phase of level 1.
We ran a single GLM-5.2 run through a serverless deployment on Fireworks AI with a $100 budget. The run showed measured local experimentation and capable backup-based recovery, which it needed after the most severe promotion failure we observed, losing 113 embedded functions and having to restore them before working again. One run is not a characterization of the model, but it is enough to temper claims of GLM-5.2 performing at the level of the frontier completers.
Opus 4.7 and 4.8 advanced further, showing capable technical correction and evidence-responsive revision, including a strong example of converting a speculative relationship into a structural test and rejecting it. Their recurring limitation was milestone overpromotion, a familiar eagerness to convert favorable local results into readiness claims before visible defects were resolved. They required operator challenge at important quality boundaries and did not sustain closure through the full benchmark. We are not altogether certain these newer iterations are in fact performing better than the earlier Opus 4.6 for our use cases.
The common pattern across this tranche was local competence without dependable project-wide closure. These models performed many of the individual behaviors seen in the strongest runs. What they did not demonstrate was the same behavior recurring through the full eight-stage investigation.
GPT-5.6 Sol | A New Class of Contender
When GPT-5.6 Sol was introduced, we seized the opportunity to evaluate it against our real-world benchmark. As the name implies, we expected incremental improvement from GPT-5.5 but instead found the model operating in a league of its own, with all three GPT-5.6 Sol runs completing the benchmark at different effort levels.
Sol was far from infallible. Its runs declared checkpoints clean or complete before later review found meaningful problems. The distinctive pattern was what happened next. Sol repeatedly responded to a local failure with a structural fix, reassessing the quality-control process that had let the mistake through and building scaffolding, such as codifying file hashes for even routine operations, to keep entire classes of failure from recurring.
GPT-5.6 Sol xhigh working through fast16 function categorization
The xhigh run provides the clearest example because its error began with a valid observation. fast16 contains a program of 101 byte-pattern rules that recognize locations inside an executable, capture addresses, replace code, and repair the executable metadata those changes require. Replaying that rule program across a larger software collection, Sol found that some rules fired not only in LS-DYNA, the engineering package fast16 targets, but also in MOHID and PKPM, two applications built from a similar Fortran lineage. It proposed that fast16 might be a general compatibility component rather than a tool engineered for sabotage.
The collisions were real, and we made similar observations early in our own investigation, but the explanation gave too much weight to the mere presence of matches. The model corrected course by demanding greater rigor of itself, asking whether the matches produced a meaningful modification sequence. In LS-DYNA they did. In MOHID and PKPM they produced only haphazard fragments, often triggered by generic compiler-generated code.
What first looked like broad compatibility was better explained as collateral risk from loose rules. Sol not only self-corrected but found better footing for that correction than we did. It then withdrew the compatibility hypothesis, preserved the valid measurements, restored the LS-DYNA modification mechanism as the authoritative account, and marked the rejected branch as historical provenance so later analysis would not inherit it.
5.6 Sol’s performance has pushed the boundary of the possible for automation in cyber threat investigations. Given its ready availability, we are interested in seeing how these capabilities enable the practitioner ecosystem to advance the state of the art in the same manner as alternate research disciplines like math and physics.
Project-Scale Recovery
What separated the completed runs from the rest was not technical insight, error detection, or willingness to revise. Every cohort supplied examples of each. The recurring difference was the scale at which recovery occurred. Across a long-horizon investigation, the completed runs did more than correct the sentence or script in front of them. They rebuilt the part of the project that later work would trust, in a repeatable pattern:
Withdraw the contradicted claim rather than softening its wording while continuing to rely on it.
Map the blast radius, identifying every conclusion, artifact, and test that depends on the disputed result.
Repair the cause, correcting the analysis or the quality-control gap that let the error through.
Carry the correction through the files another analyst or later stage will actually use.
Reopen the artifacts and run a check capable of disproving the corrected claim.
Separate work that blocks the current conclusion from uncertainty that can be disclosed and deferred.
That last step is easy to underestimate. An investigation can fail by stopping too early, and it can also fail by expanding forever. Deciding which unresolved questions block a conclusion is what convergence means in this benchmark.
This is not a single new reasoning faculty. It is the coordination of familiar capabilities over time, spanning correction, scope control, verification, state management, synthesis, and stopping judgment. The evidence supports project-scale recovery as a common feature of the completed runs. It does not prove a permanent intelligence threshold, and looking only at finishers cannot explain why every other trajectory fell short. The number of runs per family is small, and operator involvement was consequential throughout. What survives those caveats is the modest formulation that the completed runs repeatedly re-established a trustworthy investigative state after later evidence destabilized it.
Notes on Cost
Across all experiments, our runs consumed more than 23 billion tokens across model families and effort levels. The aggregate cache read rate was 93.58%, translating to different discounting rates across providers. Our self-hosted GLM-5.2 run offers a floor for comparison, producing 267.41M tokens on a $100 budget and a decent level-1 attempt charitably comparable to some GPT-5.5 and Opus 4.7 performance.
What These Results Mean for Analysts
The practical lesson stops well short of replacement for senior reverse engineers. The strongest runs still made serious semantic errors, accepted weak quality controls, promoted work too early, and depended on operator review at consequential moments. The new capability is that some runs remained useful after those failures. They owned substantial investigative workstreams, recovered from invalidated theories, preserved the valid parts of earlier work, and continued into publication-oriented analysis without starting over.
That changes the role these systems can plausibly play. A weaker model may be valuable as a local copilot, explaining a function, testing a hypothesis, or drafting a section. The strongest observed runs could act more like supervised investigative agents, maintaining and repairing a larger body of work across days, artifacts, and changing evidence. The supervision qualifier remains essential. Humans defined the investigative objective, controlled evidence release, exposed important blind spots, decided what quality meant, and retained final publication authority.
The scale of the shift deserves emphasis. Reverse engineering and malware analysis have been kept from scaling for three decades by the scarcity of expert practitioners. That constraint does not disappear, since expert guidance still determines whether the work can be trusted, but a single expert supervising these systems can now expand their output by any metric we track.
Conclusion
The line the benchmark exposed ran between local competence and sustained ownership of an investigation, rather than between smarter and lesser models. GPT-5.5, GLM-5.2, and the Opus 4.x family frequently produced valuable analysis. GPT-5.6 Sol also made meaningful errors. What distinguished the completed runs was their repeated ability to recover and diffuse that correction throughout the whole project, rather than just confronting their immediate context.
Having completed this stage of our evaluations, we took the best candidate produced by 5.6 Sol High and put it through an extensive adversarial refinement process using an ensemble of models to produce the highest automatable quality of IDBs and investigation artifacts. We are sharing these so other researchers can verify the work output against the sample and build on them rather than start from the raw binary. We make no claim that it is perfect, only that it is the greatest level of automated refinement the most capable models achieved through proper methodology, repeated standard enforcement, and a lot of tokens burned.
That is a less dramatic claim than a new form of autonomous intelligence. It is also more useful. For defenders, the question that matters most is whether the models can reliably preserve and advance a trustworthy investigation. Once we can count on that, it’s up to us to redefine a greater scope of impact for a new frontier of cyber threat investigations.
Compaction is a context-management pattern used across agent systems to compress prior context into a denser working state for long-running tasks.
SentinelLABS evaluated OpenAI’s native Responses API implementation against our automated malware analysis evaluation harness to measure real-world impact on task quality and cost.
Compaction reduced input tokens by ~86% with no measurable change to the aggregate evaluation score.
Our analysis found that compaction can significantly reduce the cost and noise of long-running security workflows without sacrificing task quality.
OpenAI introduced native compaction in a March 2026 engineering post describing extensions to the Responses API. However, the underlying idea is not unique to OpenAI. Anthropic, Google, and other agent frameworks such as LangChain all expose or document related approaches under different names.
The core problem these systems address is familiar to anyone who has built an agentic system: context accumulates faster than it stays relevant, and eventually the model is carrying more history than signal. At that point, task quality degrades and costs climb without a corresponding improvement in output.
OpenAI’s solution was to build compaction directly into the runtime so developers would not need to build custom summarization and state-carrying systems themselves. The company noted that compaction is the mechanism Codex relies on for long-running coding tasks, which positions it as load-bearing infrastructure rather than a convenience feature.
At SentinelLABS, we set out to evaluate how well OpenAI’s compaction would work for automated binary analysis, a domain with its own particular demands on agent memory and state management.
Why Malware Analysis Is a Hard Problem for Agents
Our evaluation harness gives a model access to a decompiler and asks it to complete the following:
Identify important functions and follow code paths
Interpret strings, APIs, call relationships, and data structures
Rename functions or variables based on observed behavior
Propose types or object models and explain what the malware is doing
We compare the model’s output against golden reference analysis and written reports across scoring metrics for correctness and completeness. To achieve a high score, the model needs to maintain a working theory for the slice of the binary it is analyzing, track evidence already collected, and hold open questions alongside provisional conclusions.
Malware analysis is an iterative process with a low-reward signal. A human analyst might inspect one function, learn something, pivot to another function, revise their theory, check a data structure, then return to update their original conclusion. Models do well in our evaluation where execution paths have straightforward continuity. They struggle when connections are unclear or require multiple rounds of investigation.
In observing model performance, we noticed that the agent tended to carry an increasing volume of tokens between tasks. The pattern is familiar to anyone who has run a ReAct-style agent on a non-trivial problem. Each turn adds more context until the model is dragging the full history of the run behind it, most of which stopped being useful several steps ago.
A human analyst working the same problem does not keep every raw observation equally active. They compress state between sessions. They remember that a function is probably the command dispatcher, that a particular object looks like transport state, that a given path was a dead end. They also write findings in a notebook, externalizing what they want to persist so they do not have to hold it all in working memory.
That distinction between working memory and durable memory is where compaction becomes architecturally useful.
How We Applied Compaction
Our system uses compaction to carry forward the working state: the current goal, what has already been tried, what was learned, which hypotheses remain active, what evidence changed the plan, and what questions are still open.
Specific findings and exact artifacts live outside the model context in durable storage. For malware analysis this includes logs and tool outputs, decompiled functions, intermediate artifacts, and ground-truth comparisons. When the agent needs exact evidence, it retrieves it from storage rather than relying on the compacted context to preserve it verbatim. In our use case tool use and response for binary exploration created increasingly large prompts. As the model used more tools to explore the space it added new findings – not all of them necessary. We leveraged compaction to summarize those tool calls and findings into more manageable chunks to maintain the working memory of the agent, but dramatically reduce the operational token overhead.
This split is what makes compaction measurable. A workflow that relies on compaction to preserve exact evidence will eventually produce incorrect answers when summarization or compression flattens crucial details. A workflow where compaction handles working memory and durable storage handles facts can be evaluated cleanly, because the boundary between the two is explicit.
Results
Across several long-running malware analysis agent evaluations we, compared runs with compaction enabled against runs without it.
Metric
Change
Input tokens
-86%
Output tokens
-31%
Reasoning tokens
-33%
Model calls
-1 (one fewer per run)
Aggregate evaluation score
Effectively unchanged
The token reductions were substantial. The aggregate evaluation score holding flat is what matters. We were able to carry forward enough state for the workflow to continue correctly while dramatically reducing the context processed per run.
One metric did decrease: domain object modeling, meaning the model’s ability to recover the higher-level objects and structures that explain the malware’s behavior. This is not a minor caveat. For malware analysis, object and type recovery is often where the most analytically valuable conclusions are drawn.
Our read is that compaction occasionally flattened structural reasoning that would have been useful later, and it reinforces why exact artifacts must live in durable storage rather than the compacted context.
Nevertheless, our research found that compaction made longer-running analysis practical and preserved the main evaluation outcome while doing it.
Implementation
Model providers expose compaction capabilities differently. For example, Anthropic and OpenAI both provide server-side compaction; however, OpenAI exposes an additional standalone compaction endpoint. This allows developers to solve the same problem at different points in their workflows as explained below.
Server-side Compaction
This is the simpler starting point. The Responses API call includes a compaction threshold in context_management. When context length crosses that threshold, the API compacts prior context automatically during the response, with no separate call required from the application.
This gives explicit control over when compaction happens. The application sends a context window to /responses/compact and receives a compacted context window back, which then becomes the input for the next response call.
For our malware analysis workflows, standalone compaction was useful at phase boundaries. For example, compacting after initial triage before entering deeper function analysis. This also lets you inspect metrics before and after compaction, which is useful for identifying where specific evidence is being compressed and whether that compression affects downstream scoring.
The important constraint with the standalone endpoint is to treat the returned compacted window as the next canonical context window. Do not prune it manually unless the workflow has a specific and well-understood reason to do so.
A practical decision rule for choosing between them:
Natural phase boundaries make explicit compaction useful
Chat assistant with occasional long sessions
Server-side
Low overhead
Evaluation harness measuring memory quality
Standalone
Allows direct comparison of pre- and post-compaction behavior
Workflow requiring citations or exact evidence
Neither alone
Keep artifacts in durable storage and retrieve when needed
How to Use Compaction
The main takeaway for us was that compaction works best when it is part of a broader context-engineering strategy.
Separate working memory from source-of-truth artifacts. Compaction is appropriate for the immediate state the model needs to continue working. Exact evidence belongs somewhere else. This boundary matters both for correctness and for being able to evaluate whether the compacted run behaved correctly.
Compact long-running workflows. Compaction has the most impact when a task involves many steps and repeated tool use. Short interactions have little to compress.
Start with server-side compaction. For most agent loops it is the fastest way to learn whether compaction helps. Move to standalone when compaction policy becomes part of the task or evaluation design.
Do not evaluate on cost alone. A run can become significantly cheaper while losing task quality, depending on what was compacted. Resource and outcome metrics need to be tracked together.
Preserve negative information. Long-running agents need to remember what failed, not just what worked. Failed paths carry state that informs subsequent decisions, and compaction can discard them if the workflow does not explicitly mark them as worth preserving.
Treat compaction as lossy until proven otherwise. Use evaluations, traces, and artifact comparisons to verify that the compacted run still behaves correctly. Our domain object modeling result is a reminder that what looks like clean compression can still affect specific downstream capabilities.
Conclusion
Compaction is part of a broader shift from prompt engineering to context engineering. Prompt engineering concerns what we ask of the model in a single turn whereas context engineering concerns what the model gets to see across multiple turns: what gets compressed, what gets retrieved, what gets written to durable state, and what gets discarded.
For agents running long-horizon tasks, context engineering may be as important as model selection. A strong model with poor state management will lose the thread on a complex task. A model with better context discipline may make steadier progress and complete more tasks. That tradeoff compounds quickly across the kind of multi-step security workflows we are trying to evaluate.
Without compaction, realistic long-running security agent workflows become too large, noisy, and expensive to measure cleanly. With it, the scope of what is practical to evaluate expands. We view compaction not just as a tool for making agents cheaper, but as part of the infrastructure required to evaluate whether they actually work.
The latest announcements from OpenAI and Anthropic mark another important step forward for frontier AI. They also reinforce something we’ve believed at SentinelOne® for years: the future of cybersecurity will be shaped by AI-native defense.
SentinelOne has worked closely with frontier labs for years, including OpenAI, Anthropic, and Google DeepMind, and naturally continues to do so. While we cannot always share the specifics of every collaboration, these partnerships have provided, and continue to provide meaningful insight into how advanced models are evolving and where they can create real impact across security. Many of these learnings and capabilities are already embedded in our platform, protecting customers from the most advanced attacks – every day, stopping zero day exploits no other solution is currently able to.
What stands out most is not simply that frontier models are becoming more capable, but that they are accelerating the broader shift toward faster, more intelligent, and more automated security operations. On the one hand, they are improving how the cyber industry and defenders identify weaknesses, analyze complex systems, and reason about attack paths at scale. On the other, they are giving attackers the advantage of speed and scale when it comes to finding new vulnerabilities. Progress in this race matters, but it is only one part of the broader security picture.
In practice, and without discounting the severity of uncovering exponentially more bugs in software, raw vulnerability counts rarely map cleanly to real-world risk. Many vulnerabilities are not meaningfully exploitable in live environments, and many are already reduced by architectural layers, controls, mitigations, and runtime protections. The gap between theoretical exposure and operational risk is often substantial. What matters most is the ability to understand real conditions, prioritize what matters, and stop actual attacks across complex environments, even when faced with novel threats and zero days.
That has been SentinelOne’s pioneering principle and the advantage we’ve delivered to our customers from the beginning.
From day one, SentinelOne was built to operate at machine speed, using behavioral AI, automation, and autonomous protection to detect, defend, and respond across endpoint, cloud, identity, data, network, and AI attack surfaces. As frontier AI continues to advance, the value of that approach only grows. To demonstrate our commitment to these principles, we’d provide two distinct examples.
First, in the last few weeks alone, the benefit of such an approach has played out in supply chain attacks, like LiteLLM, Axios, and CPU-Z, all illustrative of novel threats and the risk of trusted agents and workflows in the AI era. In each case, autonomous response at machine speed was the only antidote to block these novel threats that leverage unpatched, or zero day vulnerabilities.
Second, SentinelOne demonstrably expanded our own ongoing efforts to secure our technology. Along with the standard, established efforts we’ve used for years, SentinelOne has used multiple, AI-driven models to constantly examine our technology and architecture in techniques virtually identical to those discussed in Anthropic’s technical details for researchers and practitioners released April 7th 2026 (Assessing Claude Mythos Preview’s cybersecurity capabilities). This activity has been ongoing for months and is also consistently reviewed for findings as well as evaluated as a program by the SentinelOne executive team. It is our commitment to build and deliver secure technology and we do not see an effective future in this work without robust AI-driven methods, and an inclusive, multi-model approach.
As we look at the overall AI landscape, the shift is already underway, and it plays directly to SentinelOne’s strengths. The industry is moving toward more autonomous, more adaptive, and more intelligence-driven security. That is the future we helped pioneer, and one we are uniquely positioned to lead.
Our clear advice to defenders: Invest in machine speed defense and visibility right now. Ensure your defenses are up to date and well configured. Ground yourself in true research, not press releases and hype. As an example, many of the press and information shared by third parties around Anthropic’s new model release have lacked any substantive data – in many cases those statements preceded any real, tangible experience with the preview models in question. Inversely, the AI Security Institute (AISI) released a detailed research evaluation of relevant models, which sheds light on the state of frontier AI, exploitation rates, and potential real world implications. It clearly shows the trajectory, even from older models, had been apparent for a while, and that capability has existed and in many cases has been a function of compute scaling, as well as potentially the result of looser guardrails allowing more effective compute and reasoning than guardrailed models:
Source: AISI, Our evaluation of Claude Mythos Preview’s cyber capabilities, April 13th 2026Source: AISI, Our evaluation of Claude Mythos Preview’s cyber capabilities, April 13th 2026
The AI Security Institute also goes forward and outlines the following implications:
“Mythos Preview’s success on one cyber range indicates that it is at least capable of autonomously attacking small, weakly defended and vulnerable enterprise systems where access to a network has been gained. However, our ranges have important differences from real-world environments that make them easier targets. They lack security features that are often present, such as active defenders and defensive tooling. There are also no penalties for the model for undertaking actions that would trigger security alerts. This means we cannot say for sure whether Mythos Preview would be able to attack well-defended systems.
In a regime where attackers can direct and provide network access to models to conduct autonomous attacks on poorly defended systems, cybersecurity evaluations must evolve. As capabilities continue to improve, evaluation environments that lack defenses will no longer be challenging enough to discriminate between the capabilities of the most cyber-capable models or assess trends. Our future work will involve evaluating capabilities using ranges simulating hardened and defended environments, including ranges with active monitoring, endpoint detection and real-time incident response. We will also be tracking how AI-enabled vulnerability discovery and penetration testing campaigns perform on real-world systems.”
Stay safe,
The SentinelOne team
Third-Party Trademark Disclaimer:
All third-party product names, logos, and brands mentioned in this publication are the property of their respective owners and are for identification purposes only. Use of these names, logos, and brands does not imply affiliation, endorsement, sponsorship, or association with the third-party.
Expose the AI & Cloud Secrets That Put Your Data & Systems at Risk
This report draws on 11K+ customer environments. It shows how AI and cloud adoption are increasing secrets exposure and putting data at risk.
AkiraBot is a framework used to spam website chats and contact forms en masse to promote a low-quality SEO service.
SentinelLABS assesses that AkiraBot has targeted more than 400,000 websites and successfully spammed at least 80,000 websites since September 2024.
The bot uses OpenAI to generate custom outreach messages based on the purpose of the website.
The framework is modular and sophisticated compared to typical spam tools, employing multiple CAPTCHA bypass mechanisms & network detection evasion techniques.
Overview
Whenever a new form of digital communications becomes prevalent, actors inevitably adopt it for spam to try to profit from unsuspecting users. Email has been the perennial choice for spam delivery, but the prevalence of new communications platforms has expanded the spam attack surface considerably.
This report explores AkiraBot, a Python framework that targets small to medium sized business website contact forms and chat widgets. AkiraBot is designed to post AI-generated spam messages tailored to the targeted website’s content that shill the services for a dubious Search Engine Optimization (SEO) network. The use of LLM-generated content likely helps these messages bypass spam filters, as the spam content is different each time a message is generated. The framework also rotates which attacker-controlled domain is supplied in the messages, further complicating spam filtering efforts.
The bot creator has invested significant effort into evading CAPTCHA filters as well as avoiding network detections by relying on a proxy service generally marketed towards advertisers–though the service has had considerable interest and use by cybercriminal actors.
AkiraBot is not related to the ransomware group Akira; this name was chosen due to the bot’s consistent use of domains that use “Akira” as the SEO service brand.
Script Execution and Website Feature Targeting
SentinelLABS identified several archives containing scripts related to this framework with file timestamps dating back to September 2024. The oldest archive refers to the bot as Shopbot, likely a reference to its targeting of websites using Shopify. As the tool evolved, the targeting expanded to include websites built using GoDaddy and Wix, as well as generic website contact forms, which includes websites built using Squarespace, and likely other technologies. These technologies are primarily used by small- to medium-sized businesses for their ease in enabling website development with integrations for eCommerce, website content management, and business service offerings.
There are many versions of this tool with file timestamps in the archives indicating activity between September 2024 to present. Each version uses one of two hardcoded OpenAI API keys and the same proxy credentials and test sites, which links the archives despite the disparate naming conventions. We identified AkiraBot-related archives that had the following root directory names:
bubble_working_clone
fingerprints-server
GoDaddy
NextCaptcha and FastCaptcha
NextCaptchaBot-v6
override
petar_bot
shopbotpyv2
SHOPIFY_SYSTEM_UPDATED
updatedpybot
wix
wixbot
WORKING_FOLDER
Additionally, logs from the tool reveal that the operator ran it from the following paths, suggesting that they are most likely using Windows Server systems based on the Administrator username being the most prevalent:
C:/Users/Administrator/Desktop/
C:/Users/Administrator/Downloads/
C:/Users/Usuario/Desktop/ - only appears in the archive named GoDaddy
Originally, AkiraBot spammed website contact forms enticing the site owner to purchase SEO services. Newer versions of AkiraBot have also targeted the Live Chat widgets integrated into many websites, including Reamaze widgets.
_submit_old_website function in v14.py
The bot has a GUI that shows success metrics and lets the operator choose a target list to run against. The GUI lets the operator customize how many threads are running at once, a feature the bot uses to target many sites concurrently.
AkiraBot GUI
Spam Message Generation
Searching for websites referencing AkiraBot domains shows that the bot previously spammed websites in a way that the message was indexed by search engines.
Google search results containing useakira[.]comSpam comment on website from 2023 and content from AkiraBot templates.txt file
AkiraBot creates custom spam messages for targeted websites by processing a template that contains a generic outline of the type of message the bot should send.
Spam message template
The template is processed by a prompt sent to the OpenAI chat API to generate a customized outreach message based on the contents of the website. The OpenAI client uses model gpt-4o-mini and is assigned the role “You are a helpful assistant that generates marketing messages.” and the prompt instructs the LLM to replace the variables <WEBSITE_NAME> and <KEYWORD> with the site name provided at runtime.
AI Chat prompt from v10.py
The <KEYWORD> is generated by processing the {context} variable, which contains text scraped from the targeted website via BeautifulSoup, a library that transforms raw HTML code into human–or LLM–readable text.
AkiraBot’s generate_message function
The resulting message includes a brief description of the targeted website, making the message seem curated. The benefit of generating each message using an LLM is that the message content is unique and filtering against spam becomes more difficult compared to using a consistent message template which can trivially be filtered.
Logged AI-generated outreach messages in submissions.csv
CAPTCHA Bypass & Network Evasion Techniques
CAPTCHA Bypass
AkiraBot puts significant emphasis on evading CAPTCHAs so that it can spam websites at scale. The targeted CAPTCHA services include hCAPTCHA and reCAPTCHA, including Cloudflare’s hCAPTCHA service in certain versions of the tool.
We identified an archive with files for CAPTCHA-related servers and browser fingerprints, which allow the bot’s web traffic to mimic a legitimate end user. The archives contain a fingerprint server that runs on the same system as the other AkiraBot tools and intercepts the website loading processes using Selenium WebDriver, an automation framework that simulates user browsing activity.
The inject.js script injects code into the targeted website’s Document Object Model (DOM) which enables the tool to modify how the website loads in real time and change behaviors. inject.js manipulates values in the session via a headless Chrome instance that makes the session appear like an end user’s browser to the webserver. The script modifies multiple browser attributes that webservers use to identify the nature of the browser viewing the website, including:
Audio Context and Voice engines, which are used to profile whether a session is headless or a real browser
Graphics rendering, including canvas and WebGL attributes
Installed fonts
Navigator objects, which provide a wealth of profiling information, such as browser type, operating system & architecture, geolocation, hardware details, languages installed, and browser privacy settings
System memory, storage, and CPU profile
Timezone
The bot uses several CAPTCHA bypassing services, including Capsolver, FastCaptcha, and NextCaptcha, which are failover services for when browser emulation is insufficient to interact with the targeted website.
FastCaptcha token generator function in v10.py
AkiraBot also runs a headless Chrome instance to refresh values for Reamaze tokens periodically. Reamaze provides websites with customer support chat integrations, making this another targeted feature. The service also offers spam filters for chats on its platform, indicating that this is a known vector for spam attacks.
Reamaze token handling function
Network Evasion Techniques
AkiraBot uses many different proxy hosts to evade network detections and diversify the source of where its traffic comes from. In each archive SentinelLABS analyzed, AkiraBot used the SmartProxy service. SmartProxy’s website claims that its proxies are ethically sourced and that they provide data center, mobile, and residential proxies. Each version of the bot uses the same proxy credentials, suggesting the same actor is behind each iteration.
get_random_proxy function in The_NextCaptcha_Bot.py
While SmartProxy is a service that seems to operate within legal boundaries, it is worth noting that it has regularly had the attention of cybercriminals. The BlackBasta ransomware leaks referenced an exchange of SmartProxy credentials, for example.
SmartProxy credentials from BlackBasta leaks
Logging & Success
AkiraBot logs its spam progress to submissions.csv, which sometimes includes the AI-generated spam message contents as well. The submissions.csv file from the January 2025 archives show more than 80,000 unique domains that were successfully spammed. The script also logs failed attempts in failed.txt and failed_old.txt. The January 2025 archives showed that only 11,000 domains had failed, including previous runs of the tool. We analyzed all submissions.csv files; deduplicating the results revealed that more than 420,000 unique domains were targeted in total.
Two versions of AkiraBot used a Telegram bot for logging success metrics. The scripts monitor.py and monitor_random.py would collect success metrics from the bot and post them to a Telegram channel via API.
Telegram sending functionality in monitor.py
Telegram Detail
The Telegram functionality, contained in the monitor.py and monitor_random.py scripts, is tied into proxy rotation and CAPTCHA defeat features contained within the bundled JavaScript file script.js. The monitor.py script utilizes pyautogui to paste the contents of script.js into a browser developer console by scripting CTRL+SHIFT+J, followed by the paste command, eventually executing the JavaScript within the browser console.
pyautogui actions in monitor.py
The pasted and executed JavaScript is then responsible for attempting CAPTCHA refreshes and defeats on targeted URLs, reporting the status returned to a JSON file, stats.json. If a proxy rotation is required, to aid further in refreshing the CAPTCHA defeat attempts on a given URL, the monitor.py script handles this as well, rotating the used proxy though the iproxyonline service (fxdx[.]in).
Proxy rotation is generally enabled to avoid geographic or IP-based restrictions when repeatedly attempting to refresh and defeat CAPTCHAs. The Telegram status updates specifically report on proxy rotations and CAPTCHA submissions. Some versions of these scripts have the proxy rotation section commented out, indicating that it is an optional feature.
Telegram message submission + proxy rotation status in monitor.py
All of the analyzed monitor.py and monitor_random.py scripts contain the same Telegram token and chat_id combination.
Telegram bot data in monitor.py
This Telegram chat_id is associated with the following Telegram user data:
The spam messages frequently rotate the domain used, likely in an attempt to avoid detection. The oldest domain in use is akirateam[.]com, which was registered in January 2022 on a Germany-based IP, 91.195.240[.]94, without further updates until March 2023. The second oldest domain is goservicewrap[.]com, which was registered in April 2024 and resolved to 86.38.202[.]110, a Hostinger IP in Cyprus.
Several AkiraBot domains have interesting connections through historical DNS activity. The subdomain mail.servicewrap-go[.]com briefly shared a CNAME record pointing to 77980.bodis[.]com, which is associated with various malicious activities, including a 2023 malvertising campaign. This domain also received communications from several Windows executable files that were detected as various banking trojans.
An odd relationship stood out in anchor links referencing 77980.bodis[.]com: the website unj[.]digital contained anchor links from December 2024 through February 2025 pointing to 77980.bodis[.]com. UNJ Digital’s website describes itself as a digital marketing and software development firm. The subdomain smtp.unj[.]digital also has a CNAME record pointing to 77980.bodis[.]com, fortifying a connection between these hosts. While the website now highlights offering digital content services, as of late 2024 the site showed a focus on increasing marketing revenue.
Screenshot of content on unj[.]digital circa October 2024Screenshot of content on unj[.]digital circa March 2025
Akira and ServiceWrap SEO
AkiraBot uses two distinct themes in their SEO offering domain naming conventions: Akira and ServiceWrap. Reviews for both services on TrustPilot are similar: many 5-star reviews with similar, potentially AI-generated contents, and the occasional 1-star review complaining that the site is either a scam or has spammed the person leaving the review.
The 5-star reviews tend to follow a pattern where the reviewer has one previous review that was made 1-5 days before the Akira or ServiceWrap review. The review themes are very similar across these 5-star reviews, though the contents and structure are always unique. We believe the actor may be generating some fake reviews, though it is difficult to say with certainty.
Trustpilot review for servicewrapgo[.]com Trustpilot review for useakira[.]com Trustpilot review for useakira[.]com
Conclusion
AkiraBot is a sprawling framework that has undergone multiple iterations to integrate new spamming target technologies and evade website defenses. We expect this campaign to continue to evolve as website hosting providers adapt defenses to deter spam. The author or authors have invested significant effort in this bot’s ability to bypass commonly used CAPTCHA technologies, which demonstrates that the operators are motivated to violate service provider protections.
AkiraBot’s use of LLM-generated spam message content demonstrates the emerging challenges that AI poses to defending websites against spam attacks. The easiest indicators to block are the rotating set of domains used to sell the Akira and ServiceWrap SEO offerings, as there is no longer a consistent approach in the spam message contents as there were with previous campaigns selling the services of these firms.
SentinelLABS thanks the OpenAI security team for their collaboration and continued efforts in deterring bad actors from abusing their services. The OpenAI team shared the following response following their investigation:
“We’re grateful to SentinelOne for sharing their research. Distributing output from our services for spam is against our policies. The API key involved is disabled, and we’re continuing to investigate and will disable any associated assets. We take misuse seriously and are continually improving our systems to detect abuse.”