Introduction

As part of proactive threat monitoring, Kaspersky’s Cyber Threat Intelligence team identified a malicious campaign involving C2 agents controlled by artificial intelligence. Confirmed victims of the campaign included organizations in the education sector in several countries across Africa and Latin America. CyberStrikeAI played a key role in the post-exploitation phase. This is an open-source framework in which an LLM orchestrator on the command-and-control server plans actions, selects tools, and generates commands, while an implant executes them on compromised hosts. Using CyberStrikeAI, the attackers progressed from reconnaissance to full domain compromise.

According to our data, this is one of the first public descriptions of a compromise involving an LLM in the wild. The campaign we discovered clearly illustrates how the role of AI in attackers’ arsenals is evolving: until recently, language models were used primarily during preparatory stages – for writing phishing emails, debugging code, or searching for vulnerabilities. Now, however, they are being embedded into the very core of the attack, automating actions at every stage: payload execution, persistence on the host, discovery, lateral movement, and, finally, data collection and exfiltration. Post-exploitation – the stage that traditionally required manual work by an experienced operator – plays a particularly significant role here.

A key feature of the scheme using the CyberStrikeAI framework is that the implant ceases to be a simple tool for executing the operator’s commands. Decisions are distributed across two levels: the operator defines the task in general terms, while the model determines how to execute it – specifically, which commands to send to the host and in what sequence. Thus, the strategic planning remains with the human, while the choice of technical steps is delegated to the model, and the implant executes the model’s decisions.

To understand how this scheme works in practice, let’s examine the chain of actions taken by the attackers, as reconstructed by our analysts during the incident investigation.

Timeline of the attack

Let’s examine the timeline of the attack using an example from one incident within the detected campaign.

The first phase follows a standard approach: a human operator gains access to the victim’s infrastructure via a compromised SQL Server (version 16.0.1000), then collects basic information about the host, deploys tools, and conducts network reconnaissance. All of this activity takes nearly a day, followed by a two-week pause.

The second phase is fundamentally different. The first step involves deploying the CyberStrikeAI C2 agent, which receives commands from a server-based LLM orchestrator. During the second phase, the agent performs the following:

• establishing persistence and conducting local reconnaissance within 2 minutes;
• conducting domain reconnaissance over the next 5 hours;
• after a brief pause of several days, moving on to the final stage – lateral movement via SMB and WMI, mass credential dumping on hosts, and database export – with the entire operation taking no more than one hour.

We’ll discuss these stages in more detail in the next section.

Technical details

CyberStrikeAI overview

CyberStrikeAI is an open-source, modular platform written in Go for automating penetration testing. Its key feature is combining LLM-based attack orchestration and a built-in C2 server with agents (beacons) within a single server process.

The entire “intelligent” layer is isolated on the control server. C2 functions (calling reconnaissance and exploitation utilities, managing webshell connections, assigning tasks to agents, and reading results) are available to the model as MCP tools. The implant is a classic C2 agent with no AI functionality – no API keys, no calls to AI service APIs, and no autonomous logic. It periodically polls the listener server via HTTP/S, TCP, or WebSocket, executes shell commands, and interacts with the file system. To the victim, this activity looks like a typical remote-control session, where a server-side LLM orchestrator queues tasks instead of a live operator. The LLM influences how decisions are made, but not how the corresponding commands are delivered to the host.

Although CyberStrikeAI is publicly available, specific instances of the framework can be linked to specific attackers or campaigns. Detection and attribution are possible based on the following artifacts:

• Control panel: Upon launch, a self-signed TLS certificate is generated, with the framework’s name appearing in the issuer and subject fields. The certificate is created in memory at each startup, so the not_before field effectively records the time the process was launched. Accordingly, as with Cobalt Strike, the chronology of restarts can be reconstructed from the scan history.
  

  CyberStrikeAI control panel’s self-signed TLS certificate

• Implant: The build metadata (go version -m) contains a path to the main package, which includes data pointing to the built-in CyberStrikeAI generator – a temporary directory with the build identifier. This path is identical for all payloads, making it a reliable signature. Additionally, the build info field contains version control system metadata, which can be used to compare samples against one another.
• Authentication: When communicating with the server, the implant sends an X-Implant-Token header with a fixed value in the HTTP request. The token is unique to each C2 server and is embedded into the malicious payload during the build phase; therefore, it serves as both a network indicator and a sign that the samples belong to the same server.

HTTP header parameters sent by the implant to the server

It should be noted that the implant version does not reflect the server version. The implant can operate for months while the operator updates the server-side components: listener configurations and keys are preserved during updates. In the incidents we detected, we observed the use of a single agent version throughout the entire incident. In such cases, the sample’s build date may indicate the earliest possible deployment date of the CyberStrikeAI infrastructure, but it still does not provide an estimate of the compromise duration.

Execution & Privilege Escalation

The first malicious commands on the victims’ systems in the campaign under investigation originated from the sqlservr.exe process, indicating the likely initial access vector and the context in which the attackers gained the ability to execute code. It is important to note that the commands marked as having been executed prior to agent installation were executed manually by the attacker specifically within the context of this process. At this stage, we are not yet dealing with automated implant activity, but rather with manual actions by the operator.

Before launching the agent on the machines, the attackers checked for its presence using a PowerShell script, from which we can deduce its default deployment paths: C:\Windows\Temp\, C:\Windows\, C:\ProgramData\, C:\Temp. At the same time, the attacker can disguise the agent’s name as that of a legitimate system process. In the example below, it mimics svchost.exe (svchosts.exe)

Checking for the presence of the CyberStrikeAI agent on the system

Delivery and execution of the CyberStrikeAI agent

Like most C2 agents for Windows, CyberStrikeAI uses PowerShell to execute its payload. When a request is received from the operator, the commands to be executed are additionally encoded in Base64 to bypass keyword-based detection.

Decoded form of the command:

When administrative privileges were available (for example, a domain administrator account), instead of executing commands directly, the agent on remote hosts often created scheduled tasks so that the malicious payload would run with SYSTEM privileges:

An alternative method that does not use schtasks utility is to create a task via the Schedule.Service COM object:

The payload (p1.ps1) from the example above is a reverse shell that connects to the server at 185.186.245[.]144:5501:

The p1.ps1 payload and similar scripts were distributed across the network using shared network administrative folders and the WMI protocol, which we will describe in more detail in the Lateral Movement section.

Stealth

In addition to encoding commands in Base64 and obfuscating SQL/LDAP queries – which we will describe in the Collection & Exfiltration and Discovery sections, respectively – the attackers split the malicious payload into multiple files. It is noteworthy that the scripts were transmitted in this manner even before the agent was installed – that is, at a stage where human intervention is typically required – yet they clearly bear the hallmarks of being generated by an LLM.

Below is a sequential record of several script fragments written to the file d3.b64.

Subsequently, the attackers used the certutil utility to decode the resulting file and executed it with the following command:

Once assembled and decoded, the script’s contents looked like this:

The script is a tool for decrypting DPAPI secrets from the file C:\ProgramData\dsn_line.txt. This file, in turn, is populated by the following command, which searches for a DSN string containing connection settings in the SQL Server Reporting Services configuration file rsreportserver.config and copies its contents to dsn_line.txt.

It is worth noting that some of the scripts delivered and executed as described above contain comments in Chinese (##  –  researcher’s comments).

Persistence

If the CyberStrikeAI implant was successfully delivered, the first step was to establish persistence on the target system. To do this, the agent created a scheduled task named SystemUpdate, which specified the path to the implant as the action to be performed:

Analysis of the framework’s source code revealed an alternative method of persistence: by adding a value to the registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Run, where the name of the created value matches the task name (SystemUpdate).

Agent persistence on Linux systems is also implemented in two ways – by adding the entry @reboot <path_to_beacon> & to the user’s crontab (in which case the process runs at system startup), or by adding the line (nohup <path_to_beacon> &>/dev/null &) to ~/.bashrc (in which case the agent starts when an interactive bash shell is opened).

It is worth noting that the choice of a specific persistence method is delegated to the LLM connected to CyberStrikeAI. However, thanks to the human-in-the-loop (HITL) mechanism, the operator retains full control over the process: commands generated by the model may require manual approval or correction before they are executed.

Discovery

Local Discovery

Before installing the agent on a compromised host, the attackers manually gathered basic system information using a fairly trivial set of commands:

The attackers also manually checked for internet access and obtained the external IP addresses of the compromised machines using the Chinese service dnslog[.]cn. The service generates a random third-level domain, after which the attackers attempt to resolve it on the compromised machine.

If the machine has internet access, the request succeeds, and the attackers obtain the external IP address of the compromised device on the service’s page.

For reconnaissance purposes, the agent generates a more complex script designed to map out a path for further privilege escalation by compromising SCCM or exploiting msiexec (the AlwaysInstallElevated key):

Additionally, to find ways to elevate privileges locally, the agent executed scripts to inventory services running under domain accounts and tasks running under the SYSTEM account:

During their activity, the attackers periodically checked for Microsoft Defender detections:

Remote Discovery

After gathering information about the compromised host, the attackers proceeded to conduct reconnaissance within the network. During the preparatory phase of the attack, they manually collected additional information about remote hosts using the net.exe utility.

During the main phase, the agent used PowerShell’s built-in functionality to detect accessible hosts without relying on additional utilities (such as nmap or fscan):

Additionally, relevant LDAP queries to Active Directory objects helped build a network map:

Furthermore, the presence of the installed agent on remote hosts was verified using WMI:

Domain Discovery

The next step was to gather information about the victims’ domains. To this end, before installing the agent, the attackers used standard OS utilities:

However, later on, using the CyberStrikeAI implant, reconnaissance was carried out via generated PowerShell scripts:

In addition to basic domain information, the agent performed LDAP queries to gather information for Kerberoasting and AS-REP Roasting attacks:

Finally, the agent was used to check the domain for the presence of LAPS and Microsoft Azure:

To detect signs of Microsoft Azure usage within the infrastructure, the attackers also searched for the .azcopy directory – created by the AzCopy (Microsoft Azure Storage Copy) utility – in users’ home directories:

Credential Access

After successfully conducting reconnaissance and mapping the network, the attackers moved on to the next stage – dumping credentials. For this purpose, the agent used a classic set of LOLBINs and well-known third-party tools.

The agent dumped registry hives containing credentials using the reg.exe utility:

The agent dumped the lsass.exe process using the MiniDump function of the comsvcs.dll library. In doing so, the target process’s PID was used instead of its name to make static detection based on the command line more difficult:

After successfully compromising privileged domain accounts (specifically, the built-in domain administrator), the attackers also carried out DCSync and Golden Ticket attacks via the agent using a renamed version of the mimikatz utility:

Lateral Movement

To spread across the network using compromised accounts, the attackers employed utilities from the Impacket toolkit, specifically WMIExec, as indicated by the utility’s characteristic command line:

Once installed, the agent prefers to use standard OS mechanisms, such as shared network folders.

Another method the agent uses to remotely execute commands is the WMI protocol. In particular, the agent used it to distribute the aforementioned p1.ps1 reverse shell script. This involved using compromised user accounts with local administrator privileges on the target hosts:

A second method of lateral movement via WMI uses the Invoke-CimMethod cmdlet:

Command & Control

To deliver the aforementioned tools (malicious scripts, mimikatz, and the agent itself) to the initial nodes in the infection chain, the attackers used standard OS utilities:

After the agent is installed, it checks the availability of the C2 server’s ports. The implementation mechanisms here are virtually unlimited and depend on the “creativity” of the respective LLM agents connected to the server:

To download additional utilities, the CyberStrikeAI implant also used built-in utilities (LOLBINs). The tools could be hosted on the attackers’ own servers or downloaded directly from GitHub:

Collection & Exfiltration

The data collection phase deserves special attention. It demonstrates yet another capability of LLM orchestration – the automatic generation of SQL and LDAP queries tailored to a specific task.

In one incident during the campaign, the attackers deployed the CyberStrikeAI implant on an educational organization’s SQL server and collected information from the database via PowerShell. Here is an example of a command executed by the agent:

Normalized view of the executed script with comments:

Before accessing the data, the agent checked the privileges of the account under which it connected to the database:

In decoded form, the check looks as follows:

On network resources, file collection was preceded by a recursive directory scan to identify files of interest.

To interact with the file system, the implant provides a built-in interface that supports direct uploading and downloading of files. Files are transferred directly via the main encrypted communication channel with the C2 server (HTTP or TCP), and the data itself is encoded in Base64 and packaged into the agent’s standard JSON responses.

Another built-in capability of the implant is taking screenshots of the infected machine. As with other actions, the implant does not initiate this on its own: the command to capture the screen is issued by the LLM orchestrator, and the agent merely executes it and returns the result via the same control channel.

On Windows, the command for taking screenshots looks like this:

The Linux version looks like this:

In the incidents we detected, data collection was the final stage of the attack: the chain ended there, and we observed no further activity.

Attribution

The key artifact for attribution in this campaign is the implant’s build metadata. The attackers built the malicious payload inside a Git clone of the project, so the Go compiler embedded version control system information into the binary files – the hash and commit time, as well as the vcs.modified=true flag (analogous to the +dirty suffix) – indicating uncommitted local changes in the working copy. The combination of these fields forms a unique fingerprint for a specific build.

Based on this fingerprint, the samples are divided into two independent groups, in each of which the Windows implant and the Linux implant have completely identical build metadata. A match indicates that, within each group, both payloads were built from the same project working copy.

Identical build metadata for the Windows and Linux implant samples, indicating a shared build environment

However, these groups do not overlap with one another. They do not share a common C2 infrastructure, and different versions of CyberStrikeAI are deployed on the command-and-control servers. In other words, the build metadata reliably links the samples within each group but does not provide grounds for linking the two groups together.

Differences in CyberStrikeAI versions on the command-and-control servers

The available data does not allow us to link the activity to any known group of attackers, nor to state with certainty that the same operator is behind both campaigns. Two factors prevent this. First, the tool’s openness: CyberStrikeAI is publicly available, so different operators could have independently deployed it and assembled malicious payloads under similar conditions. Second, command generation by the model: the “signature” characteristic of a specific operator is largely shaped not by a human but by an LLM. On the one hand, this increases the diversity of commands: the same command, when generated multiple times, can look different. On the other hand, it strips them of their individual characteristics, which reduces the value of behavioral metrics for attribution.

However, the Chinese-language comments found in the attackers’ scripts and the use of Chinese services still suggest a connection between the activity and a Chinese-speaking operator. In addition, analysts noted the timing of the control servers’ restarts: since the self-signed TLS certificate is regenerated each time the process starts, its not_before field records the restart time, and most of these restarts occurred during business hours in the UTC+8 time zone.

The origin of the tool itself also indirectly points in the same direction: CyberStrikeAI was developed within the Chinese cybersecurity community and is well known there.

In summary

CyberStrikeAI is a telling – though not unique – example of a shift occurring within the class of C2 frameworks: the LLM orchestrator is no longer merely an add-on to reconnaissance but has become a control layer over post-exploitation. If HexStrike AI and similar tools cover the “find and break” phase, and Sliver/Mythic/Havoc cover the “establish a foothold and control” phase, CyberStrikeAI completes the kill chain: a single server-side process written in Go executes utilities via MCP, handles webshell connections, and assigns tasks to the implant, while the model analyzes the results and decides what to do next.

The key takeaway is that the introduction of AI into the attack chain does not yet present fundamentally new challenges for detection. All the “intelligent” work remains on the attacker’s side, while the compromised host is subjected to familiar actions: from reconnaissance via Kerberoasting and AS-REP Roasting to LSASS dumps, DCSync, and the Golden Ticket attacks. This is a typical, well-documented set of domain compromise techniques. The novelty of the attack lies not in the methods themselves, but in the speed and ease with which they were executed: AI orchestration has lowered the skill requirements for the operator, but has not taken the attack beyond what defenders already know and are capable of detecting. The timing is also telling: individual key phases of the activity (such as establishing a foothold, lateral movement, and obtaining credentials) took only a few hours, and the entire attack – from agent deployment to the compromise of domain administrators – was completed within 3–4 days.

However, the CyberStrikeAI architecture does not limit users to the scenario described. The same mechanisms allow for the on-demand assembly of more complex chains – including DPAPI secret extraction and attacks on SCCM and Azure IMDS – all of which are within reach of the same group of actors, with a relatively low skill barrier. As the models evolve, the gap between the complexity of orchestration and the level of techniques used will narrow, and the operator’s role will gradually shift from setting tasks to monitoring the model’s actions.

Detection by Kaspersky solutions

It may seem that attacks using AI agents require the creation of fundamentally new detection rules. Indeed, to detect the activity of an LLM agent installed on a host within the infrastructure, it is necessary to create detection logic based on the specific characteristics of the agents. Detection rules for this activity can be found using Hunt Hub with the following filter.

However, in the case of the attack described, a different logic applies. The main difference lies in the architecture of the attack itself: the LLM agent was not located within the corporate network perimeter. It operated remotely via a C2 agent, orchestrating actions, and left no traces of its presence in the infrastructure (such as specific API requests to local LLMs or anomalous patterns of resource consumption by hosts).

The key difference between this activity and an attack by an experienced human operator lies largely in speed. Tactically, technically, and methodologically, AI, for the most part, uses exactly the same tools, utilities, and techniques as a human penetration tester. It does not write unique exploits from scratch, but rather combines known methods and dynamically adapts them to the situation, handling the tools’ errors on its own.

Since the actions performed by an AI agent on the network are indistinguishable from those of a human, classic, well-tuned correlation and behavioral analysis rules are sufficient to detect it. Here are a few examples of how Kaspersky Next EDR Expert successfully detects the attack described:

• To detect the execution of a malicious script via a remote scheduled task, you need to monitor the creation of the schtasks.exe process with the /s argument. EDR Expert detects this activity using the rules create_task_on_remote_host_using_standard_tool, scheduled_task_creation_as_system_via_schtasks, and scheduled_task_with_powershell_execution.
  
• During the attack, the adversary will need access to the victim’s credentials; in the described case, one way to obtain them was through a well-known technique that uses legitimate tools to dump LSASS, specifically by calling the MiniDump function from comsvcs.dll. The EDR Expert solution detects this activity using the rules dumping_memory_with_com_services_dll and suspicious_lsass_memory_access.
  
• A significant part of the attack chain is lateral movement, which helps the attacker navigate within the infrastructure and achieve their desired objectives. In the described case, the WinRM protocol is used to initiate connections directly from PowerShell. This is a standard technique that allows the attacker to execute commands on remote machines without using additional network tunnels. The EDR Expert solution detects this activity using the rules using_powershell_for_remote_execution_via_wmi and winrm_send_command.
  

Kaspersky Anti Targeted Attack (KATA) is used to detect malicious activity in network traffic. Communication between CyberStrikeAI agents and the command-and-control server occurs without accessing AI platforms such as ChatGPT or Claude. Instead, the agent’s C2 server is an infrastructure pre-configured by the attackers, just as in attacks that do not use AI agents. Analysis of decrypted network traffic using NDR-class solutions reveals characteristic communication patterns of CyberStrikeAI agent activity. When these patterns are detected, KATA generates an alert with the verdict Backdoor.CyberStrikeAI.HTTP.C&C.

Alert regarding the detection of CyberStrikeAI communication with a C2 server

To detect post-exploitation activity, you can use not only IDS technology but also NAD (Network Anomaly Detection). It analyzes all traffic for suspicious indicators that do not correspond to a node’s typical network activity.

For example, actions performed by CyberStrikeAI during the Domain Discovery phase are detected by an NDR:NAD rule named “Scanning domain records via LDAP queries.”

NDR:NAD alert: reconnaissance via LDAP queries detected

You can learn more about NDR:NAD technology in our article on Securelist.

You can also visualize the connections between C2 addresses and files used by attackers in the Research Graph section of the Kaspersky Threat Intelligence Portal. This includes tracking the emergence of new CyberStrikeAI implants that communicate with malicious addresses.

Visualization via the Research Graph on the Kaspersky Threat Intelligence Portal

In addition, the Threat Lookup service available on the portal allows you to determine whether malicious files belong to CyberStrikeAI using attribution mechanisms based on the executable file’s code.

Demonstration of attributing a malicious implant to the CyberStrikeAI framework

Kaspersky SIEM detects all stages of an attack by the CyberStrikeAI agent. The product repository contains over 100 rules for detecting the techniques discussed in this article; we will list the most critical ones below.

Execution of various commands from the sqlservr.exe process:

• R300_04_Windows shell started by a web application

Downloading and executing a payload via PowerShell:

• R110_03_PowerShell code downloaded and executed

Creating Windows scheduled tasks:

• R099_04_Suspicious arguments when running schtasks.exe
• R300_03_Windows shell started by the scheduler

Persisting via crontab or bashrc:

• R099_06_Suspicious cron job activity
• R405_02_UNIX shell configuration modification

Rules located at the KUMA Packages/SOC package/Configuration discovery path detect the collection of information about hosts, network settings, and domains using various reconnaissance tools. Rules that aggregate detections by reconnaissance tools:

• R224_14_Windows reconnaissance activity
• R224_09_System information discovery by running multiple utilities within a short period
• R224_04_System information discovery via standard Linux tools

Data collection for Kerberoasting and AS-REP Roasting:

• R034_01_Multiple TGS ticket requests (Kerberoasting)
• R452_18_AS-REP Roasting

Dumping registry hives containing credentials using the reg.exe utility:

• R230_02_Copying data containing credentials from the Windows registry

Dumping the lsass.exe process using the MiniDump function of the comsvcs.dll library:

• R233_04_Process memory dump via comsvcs.dll
• R262_Suspicious access to the LSASS process

DCSync and Golden Ticket attacks using the renamed mimikatz utility:

• R073_DC Sync attack
• R450_41_Golden Ticket Attack
• R100_03_Traces of Mimikatz use via cmd

Lateral movement using WMIExec:

• R460_08_Lateral movement using WMIexec Impacket
• R460_09_Lateral movement using WMIExec and Impacket

For the rules to work, you must configure the SIEM system to receive the relevant events:

• Process launches (Windows security log – 4688, auditd – execve) and PowerShell commands (4104)
• User logins to the system (Windows security log – 4624, 4768, 4769)
• Various object manipulations (Windows security log – 4662, 4663, 4656; auditd – openat)
• Interaction with shared resources (Windows security log – 5145)

Indicators of compromise

File-based indicators:

Network indicators:

A modern organization protects its information systems with a whole suite of technologies that generate thousands of security alerts every day. Looking at these alerts in bulk, you’ll see that a significant share are false positives. Ideally, these alerts would be filtered out automatically so that analysts could focus on the ones that may point to a real incident. However, writing filters is a time-consuming task in itself. To create a correct filter, an analyst needs to:

• Review the logs
• Identify the source of the activity
• Define filtering boundaries
• Choose filtering criteria
• Write a rule
• Check that the filter is robust

This post looks at how filter creation itself can be automated to reduce analyst workload.

Filter validity criteria

In the context of security alerts, a filter is a combination of event field values that marks a detection as a false positive and not worth the analyst’s attention. Filters can be generated for any combination of event fields, but not every combination yields a valid filter. That’s why we need to define validity criteria: conditions showing that a specific filter actually removes noise and doesn’t suppress useful alerts.

Detection rules in security systems typically contain a set of conditions, and each of them can generate an alert. A specific event triggers a given fragment of a detection rule. The set of events that a given fragment responds to is what we’ll call its context. A good filter should cover the entire context where possible, without going beyond it. Filters that are too broad or too narrow are useless. For example, a filter based solely on PowerShell would exclude too many events, including some that deserve the analyst’s attention. Most of the filtered events would also be unrelated to the rule in question. A filter on a specific ID, on the other hand, would cover only one event, which may never recur.

Problems in filter generation

We can run into a number of problems when creating filters. First, events often contain an excessive number of fields (around 400 on average), and most of them aren’t needed for filtering. At the same time, generating and ranking combinations of these fields is also fairly resource-intensive. So, the first step is to discard the irrelevant fields and keep only those worth using in the filter. For example, out of 400 fields in an event, four may be enough for filtering.

Another problem is string normalization. Field values (for example, the path to the same application on different systems) can vary depending on system settings, user folder names, and so on. For a filter to account for all these variations, the strings must be normalized. For file paths, in particular, you can replace the drive letter and user folders with standard environment variables and convert all paths to a single case.

Finally, if a filter completely mirrors the query of a detection rule, it effectively disables that rule. To fix this, remove the event fields used in the detection rule from the filter. For example, if a rule triggers on a file path, there are two ways to filter out unwanted alerts:

• In the filter, use fields other than the file path.
• Use a part of the file path that the rule doesn’t use. For example, for the rule file_path: ".../something.exe", you can set up a filter that excludes events with the user/unique_files/ directory in the file path.

Filtering security alerts with an algorithm

Given the problems described above, automatically generating filters for security alerts should consist of the following steps:

1. An analyst receives a security alert that they assess as a false positive and uses the filter generator.
2. The generator takes the 400-field event from the alert, along with the rule that triggered on it, and determines the context.
3. The generator selects the fields that are relevant to the filter.
4. The generator normalizes the strings.
5. The generator builds different combinations of the selected fields. To do this, we applied the FP-Growth (frequent pattern growth) algorithm to the context determined earlier. FP Growth builds a tree of frequently occurring items (fields) from a dataset (in our case, the context) and discards items that aren’t frequent enough. This reduces the number of combinations to check, which lowers the cost of finding the most relevant ones.
6. The generator ranks the resulting combinations and offers the analyst the top three to choose from.

For example, consider an event described by the following JSON:

After generating and ranking the combinations, we got three filters based on file paths:

• processcmdline:("nodejs\\node.exe" AND "*****\\appdata\\roaming\\npm\/node_modules\/pi-coding-agent\/dist\/cli.js")

• file_path:"*****/.pi\\agent\\bin\\fd.exe"

• processcmdline: "programfiles\\nodejs\\node.exe"

The last filter turned out to be too broad, but the analyst marked the other two as valid.

Pros and cons of the algorithmic approach

The algorithmic approach to filter generation has several advantages. First, it’s simple and deterministic: the same alert always produces the same result. Second, it’s interpretable, meaning we can see why a given combination ranked near the top of the list and why a particular field didn’t make it into the list of candidate fields at all. Third, the stages of the process are independent of each other, so individual steps can be improved and tuned without the risk of breaking the algorithm elsewhere. Finally, it has no external dependencies or models, which makes it easy to deploy in any customer environment.

However, it also has drawbacks. The first is its dependence on context. If there’s no context for some reason — for example, the rule’s first-ever trigger — a filter can’t be created. The second problem is string normalization. There’s a high chance of running into atypical cases that the normalization rules don’t cover. Often such cases can’t be described correctly at all, because they contradict each other.

These problems can be solved with large language models (LLMs). When an LLM generates the filters, no context is needed, because the LLM doesn’t iterate over or rank combinations. The LLM also takes a more flexible approach to string normalization, including non-standard cases. That said, an LLM doesn’t have to be involved at every stage of the generation process. We decided to keep the algorithmic solution where it works well. For example, a simple algorithm selects suitable fields without errors, so we didn’t move this stage to the model.

Filter generation with an LLM

The LLM-based filter generation looks like this:

The input is still an alert containing an event with 400 fields, plus the rule that triggered on that event. Then an algorithm selects the most suitable fields for the filter. Next, the LLM processes the input — the event, the selected fields, and the detection rule — and generates three filter options along with an explanation for them. The explanation helps the analyst understand why these particular filters were suggested and make an informed choice.

Consider the following event in JSON format:

The LLM suggested three filter options that all contain the same command line fragment:

• file_path: "\\WindowsPowerShell\\powershell.exe" AND filecmdline: "Get-VM -Name kafka\*"

• filecmdline: "Get-VM -Name kafka\*"

• filecmdline: "kafka*"

As in the previous example, two of the three output filters were valid. The last one was too broad to be usable. The analyst chose the first filter with minor edits:

file_path: "powershell.exe" AND filecmdline: "Get-VM -Name kafka\*"

Pros and cons of generating filters with an LLM

Using an LLM solved the problems of purely algorithmic filter generation: dependence on context and errors in string normalization. However, the single-request approach had its own drawbacks. First, it had no output validation. The LLM returned filters as-is, with no syntax check and no way to correct errors. Second, unlike in the algorithmic version, all filter generation tasks were handled by a single system prompt. As a result, a minor fix in one place could break the logic in another.

Using multiple AI agents

To address the problems of generating filters with a single model and a single system prompt, we tried distributing the tasks across different LLM agents. In this version, we brought in agents as early as the stage of selecting relevant fields. The chain includes four agents in total:

• Selector: selects the relevant fields.
• Generator: generates filters.
• Validator: evaluates the generated filters. If the filters are found to be invalid, it returns the task to the generator. If the selected fields are found to be irrelevant, it starts a new field selection cycle. If the filters pass validation, or the limit on regeneration attempts is reached, the valid filters (or the best ones available) are passed to the analyst.
• Commentator: analyzes the output filters and explains which field values were used and why.

For the following JSON event, the agents produce three valid filters based on the file path.

The filters differ in the additional parameters that support the detection:

• file_path:"sysvol\\domain\\scripts\\logon.bat"

• file_path:"sysvol\\domain\\scripts\\logon.bat" AND file_md5:"0x7..6E"

• autorun_entry: "Group Policy\\State" AND file_path: "SYSVOL\\domain\\scripts\\logon.bat"

Pros and cons of the agent-based approach

LLM agents give us a system that can validate and correct filters. Its modular architecture also lets us tune and improve different stages of generation independently. However, this approach makes more LLM calls than the previous ones, so the system is slower. It also depends on access to a model, which may not be available in the organization’s isolated environment.

Evaluating the filter generation system

To evaluate how well the filter generation system works, we use two main metrics:

• Approval rate. For each security alert, the system suggests three filters. If the analyst selects at least one of them, we count the generation as successful. For example, in the case below, the approval rate is 2/3.
  
• False positive reduction rate (FP drop). This metric shows how much a filter helps reduce the number of false positives. It can only be measured over time, because we need to compare the number of false positive alerts received during a set period before and after the filter is deployed.

Below is a comparison of the three filter generation approaches by these metrics:

As the results show, LLMs significantly improve the efficiency of filter generation. At the same time, the LLM agentless and agent-based approaches perform almost identically. So we chose the simpler and faster option: an LLM with a single system prompt. As a result, we cut false positives by 99%, increased the number of filters generated per day by 28%, and made analysts’ alert handling 17% more efficient.

Takeaways

Practice has shown that filter generation can be automated effectively. We’ve looked at several options for automatically generating filters for false-positive security alerts: a purely algorithmic approach, an LLM-based approach, and an approach built on a group of LLM agents. All of them can generate valid filters and make the analyst’s work much easier, but each has its own pros and cons.

The algorithmic approach scored lower on the evaluation metrics, but in two-thirds of cases, it still produces filters that satisfy the analyst. It works especially well when there’s already a set of similar events to filter out, meaning enough context has accumulated. The algorithmic generator can also be deployed in any environment. Both standalone LLM and multi-agent LLM approaches show high accuracy, and their results differ only slightly, so the slower agent-based approach looks excessive at this stage. If an LLM is available, the best option appears to be a single model that handles several stages of the generation cycle.

It’s no secret how artificial intelligence (AI)-powered tools and systems, which promise users exponential productivity, growth, and efficiency, have quickly become integral in organizations' processes and workflows. AI — specifically generative AI (GenAI) and large language models (LLMs) — is now a foundational element of the software development process, allowing developers to build and scale software at a velocity that would have been impossible a mere decade ago. These technologies have permeated the software development lifecycle (SDLC), from ideation to maintaining software, ensuring that concepts are translated swiftly into code.

Update Oct 22 2026: New sections were added for “Observed Threat Activity and Indicators” and “Mitigations and Recommendations.”

The vulnerability landscape shifted significantly in Q2 2026. First, the number of registered CVEs reached an unprecedented level. This is driven primarily by the widespread adoption of AI, both for application development and search for security flaws. This resulted in entire new classes of vulnerabilities emerging, particularly in the Linux networking subsystem.

Second, security researchers have been publishing exploits for unpatched vulnerabilities more frequently. Publications like these can generate significant fallout, since they potentially open the door for attackers to target unprotected systems.

Statistics on registered vulnerabilities

This section provides statistical data on registered vulnerabilities. The data comes from Kaspersky’s vulnerability knowledge base, which draws on the CVE database as well as the Russian BDU database and GitHub Advisory (GHSA). As a result, the figures for previous reporting periods may differ from those published in earlier reports.

We examine the number of registered vulnerabilities for each month over the last five years. As the chart below shows, this number continues to surge, a trend reflected across all the databases we track. It’s driven primarily by the widespread adoption of AI tools: as we predicted in our previous report, these tools have played a major role in the discovery of vulnerabilities in third-party software. Meanwhile, these tools often contain security issues of their own. For example, OpenClaw, a popular AI project, ranked 12th among those with the highest number of vulnerabilities discovered and published in Q2, with over 200 CVEs registered during the reporting period. Finally, AI development tools are also contributing to the vulnerability landscape, since the quality of the code they produce can vary widely. Therefore, the rate at which new vulnerabilities are discovered will inevitably keep growing.

Total published vulnerabilities per month from 2022 through 2026 (download)

Next, we analyze the number of new critical vulnerabilities (CVSS > 9.0) over the same period.

Total critical vulnerabilities published per month from 2022 through 2026 (download)

As the chart shows, the number of published critical vulnerabilities jumped sharply in Q2. This is because using AI for vulnerability research makes it possible to analyze massive amounts of previously unexamined code, uncover new attack surfaces, and identify entire classes of vulnerabilities that have gone unnoticed for decades. In particular, AI was used to find a series of Dirty Frag vulnerabilities in the Linux kernel.

Exploitation statistics

This section presents statistics on vulnerability exploitation for Q2 2026. The data draws on open sources and our telemetry.

Windows and Linux vulnerability exploitation

Q2 2026 saw a new precedent in the publication of vulnerabilities in Windows components and exploits for these: researchers no longer waiting for CVE registration, let alone patches. A case in point: a researcher who goes by Nightmare Eclipse (also known as Chaotic Eclipse) published a list of new “named” vulnerabilities across various Windows subsystems. At the time the technical details were published, none of the vulnerabilities had been assigned a CVE identifier:

• BlueHammer: a local privilege escalation vulnerability in Windows Defender. During signature database updates, a time-of-check to time-of-use (TOCTOU) race condition occurs, allowing an attacker to substitute the directory where temporary update files are written. The researcher published a fully functional exploit for the vulnerability.
• RedSun: another logical vulnerability in Windows Defender with a working exploit. Suspicious and malicious files marked as “cloud” can be overwritten or restored to their original directory with elevated privileges. The exploit incorporates fragments of algorithms that make it possible to leverage various logical vulnerabilities in Windows, effectively combining a large number of popular exploitation techniques.
• YellowKey: a vulnerability that lets the user bypass BitLocker full-disk encryption and access system data through the Windows Recovery Environment (WinRE). A fully functional exploit was also published.
• GreenPlasma: a vulnerability that enables system object injection via the CTF loader for the Collaborative Translation Framework (CTFMON) service in Windows. The original publication included an exploit with limited functionality.
• RoguePlanet: yet another Windows Defender vulnerability that, like BlueHammer, stems from a TOCTOU issue, this time in the engine responsible for real-time system scanning. The published exploit uses the vulnerability to overwrite the system file wermgr.exe with a malicious one.
• UnDefend: another vulnerability in the Windows Defender service. This time, the exploit causes a denial of service and blocks updates.

Even though such cases remain isolated for now, we believe they’ll grow into a full-fledged trend. Early publication of exploits gives attackers an advantage over software developers, who are left with no time to fix the issues.

Veteran vulnerabilities in Windows software also remain relevant. These are the ones our solutions most frequently detect exploits for:

• CVE-2018-0802: a remote code execution (RCE) vulnerability in the Equation Editor component
• CVE-2017-11882: another RCE vulnerability also affecting Equation Editor
• CVE-2017-0199: a vulnerability in Microsoft Office and WordPad that allows an attacker to gain control over the system
• CVE-2023-38831: a vulnerability in WinRAR that involves improper handling of objects within an archive
• CVE-2025-6218 (formerly ZDI-CAN-27198): another WinRAR vulnerability allowing the specification of relative paths to extract files into arbitrary directories, potentially leading to malicious command execution
• CVE-2025-8088: a vulnerability similar in exploitation method to CVE-2025-6218. The attackers used NTFS Streams to circumvent controls on the directory into which files are being unpacked

The vulnerabilities listed here can be leveraged to gain initial access to a vulnerable system and for privilege escalation. This underscores the critical importance of timely software updates.

That said, the number of Windows users who encountered exploits declined slightly in Q2, hitting an 18-month low.

Dynamics of the number of Windows users encountering exploits, Q1 2025 – Q2 2026. The number of users who encountered exploits in Q1 2025 is taken as 100% (download)

Linux also hit a rough patch in Q2 2026. Specifically, the period saw the disclosure of the Dirty Frag family of vulnerabilities, which lets an attacker reliably escalate privileges within the operating system.

All the vulnerabilities published in Q2 2026 were, in one way or another, related to the Linux caching subsystem. Here are the ones being most actively exploited:

• CVE-2026-31431 (Copy Fail): a local privilege escalation vulnerability in the Linux kernel that lets an unprivileged user modify the page cache and gain root privileges. Especially dangerous for cloud and containerized environments
• CVE-2026-43284, CVE-2026-43500 (Dirty Frag): a family of vulnerabilities in the Linux networking subsystem (IPsec ESP and RxRPC) that lets a local user overwrite the page cache and escalate privileges to root
• CVE-2026-46300 (Fragnesia): a local privilege escalation vulnerability in the Linux kernel related to packet fragment handling and the page cache mechanism. It lets an unprivileged user gain root privileges and is also classified as part of the Dirty Frag family
• CVE-2026-31635 (DirtyDecrypt): a Linux kernel vulnerability that lets a local attacker escalate privileges due to improper handling of decryption operations and page cache data modification
• CVE-2026-43494 (PinTheft): a Linux kernel vulnerability that lets a local user gain elevated privileges due to errors in the memory page pinning mechanism
• CVE-2026-46331 (pedit COW): a vulnerability in the Linux kernel’s traffic control subsystem (tc-pedit) that exploits a flaw in copy-on-write to modify the page cache and subsequently escalate privileges to root

The vulnerabilities described above were quickly embraced by attackers. At the same time, our solutions continue to detect exploitation attempts targeting older vulnerabilities as well:

• CVE-2022-0847: a vulnerability known as Dirty Pipe, which enables privilege escalation and the hijacking of running applications
• CVE-2019-13272: a vulnerability caused by improper handling of privilege inheritance, which can be exploited to achieve privilege escalation
• CVE-2021-22555: a heap out-of-bounds write vulnerability in the Netfilter kernel subsystem
• CVE-2023-32233: another Netfilter subsystem vulnerability that allows for Use-After-Free conditions and privilege escalation through improper processing of network requests

Dynamics of the number of Linux users encountering exploits, Q1 2025 – Q2 2026. The number of users who encountered exploits in Q1 2025 is taken as 100% (download)

In Q2 2026, the number of Linux users who encountered exploits declined slightly compared to Q1. Given that a significant share of new vulnerabilities are tied to the operating system’s caching subsystem, we recommend installing patches as quickly as possible, or disabling vulnerable kernel modules if patching isn’t an option.

Most common published exploits

The distribution of published exploits by software type in Q2 2026 includes categories that haven’t appeared in the sample for a long time. For instance, we’re once again seeing exploits targeting SharePoint. It’s worth noting that while several vulnerability write-ups for Exchange and SharePoint were published during the quarter, most turned out to be fake, AI-generated research. While the articles and exploit source code themselves look fairly polished, they describe nonexistent problems in the software or its components — often close to genuinely vulnerable mechanisms — in order to mislead researchers. This type of attack is aimed at increasing the time it takes to detect real vulnerabilities. In some cases, the description of a nonexistent vulnerability came bundled with completely unrelated malware.

Distribution of published exploits by platform, Q1 2026 (download)

Distribution of published exploits by platform, Q2 2026 (download)

Vulnerability exploitation in APT attacks

We analyzed which vulnerabilities were exploited in APT attacks during Q2 2026. The rankings provided below include data based on our telemetry, research, and open sources.

TOP 10 vulnerabilities exploited in APT attacks, Q2 2026 (download)

In Q2 2026, a trend emerged in APT attacks toward exploiting new vulnerabilities right from the moment they’re published. As before, we’re also seeing a large number of zero-day vulnerabilities. The Langflow vulnerability deserves particular attention: it’s one of the first cases of an APT group exploiting AI technology, which many organizations are only just beginning to integrate. Because most of this tech is proprietary, it has a considerable number of security blind spots. Therefore, given the growing number of AI-based automation tools, we strongly recommend going beyond the usual patching and developing secure procedures for credential use and sensitive data handling in systems that rely on agents and LLMs.

C2 frameworks

In this section, we examine the most popular C2 frameworks used by APT groups and analyze the vulnerabilities targeted by the exploits that interacted with C2 agents in APT attacks.

The chart below shows the frequency of known C2 framework usage in attacks during Q2 2026, according to open sources.

TOP 10 C2 frameworks used by APTs to compromise user systems, Q2 2026 (download)

Sliver, Havoc, AdaptixC2, and Metasploit remain the most widely used C2 frameworks. After studying open sources and analyzing samples of malicious C2 agents that contained exploits, we determined that the following vulnerabilities were utilized in APT attacks involving the C2 frameworks mentioned above:

• CVE-2026-35273: a vulnerability in Oracle PeopleSoft PeopleTools that security vendors classify as server-side request forgery (SSRF). The details of the vulnerability have never been disclosed, although some research covers the post-exploitation steps
• CVE-2023-46604: an insecure deserialization vulnerability in Apache ActiveMQ that allows arbitrary code execution in the context of the service process
• CVE-2024-12356 and CVE-2026-1731: command injection vulnerabilities in BeyondTrust software that allow an attacker to send malicious commands even without system authentication
• CVE-2023-36884: a vulnerability in the Windows Search component that allows commands to be run on the system, bypassing the mark-of-the-web (MoTW) mechanism
• CVE-2025-53770: an insecure deserialization vulnerability in Microsoft SharePoint that allows for unauthenticated command execution on the server
• CVE-2025-8088 and CVE-2025-6218: similar directory traversal vulnerabilities in WinRAR that allow files to be extracted from an archive to a predetermined path, potentially without the archiving utility displaying any alerts to the user

These vulnerabilities show that attackers used them for initial access and privilege escalation on vulnerable systems, setting the stage for launching a C2 agent. They include both zero-day vulnerabilities and fairly well-known security issues.

LLM/AI tool vulnerabilities

This section analyzes data published in Kaspersky’s vulnerability knowledge base. We reviewed the Q2 2026 version of the knowledge base.

As mentioned above, AI tools, plugins, and technologies have proven fairly effective at automating the search for problematic code and anomalous behavior. The high speed at which new vulnerabilities are being discovered has naturally created a need to fix them just as quickly. AI is often used for this too, which increases the volume of code being generated. However, neither code written without human involvement nor AI-generated advice is always correct.

The chart below covers registered vulnerabilities in AI tools for 2025–2026.

Number of published vulnerabilities in LLMs, AI tools, and plugins with similar functionality, 2025–2026 (download)

As the charts show, AI tools are racking up a substantial number of registered vulnerabilities, and that number keeps growing quarter over quarter. It’s also worth looking at how AI tool vulnerabilities break down by type, according to the CWE system:

TOP 6 vulnerability types in products that implement or use AI/LLM logic, 2025–2026

Interestingly, vulnerabilities of an undetermined type have ranked first in every quarter since the start of 2025. Traditionally-made software has the same issue, and it doesn’t look like the growing number of AI tools will fix it. It’s also notable that the list includes classes CWE developers themselves don’t recommend using for vulnerability classification, since they lump together a whole range of more specific types. CWE-284 is an example of this.

Looking at the most common classes, the key issues found in AI-related software can be summed up as follows:

• Inadequate access control over critical system objects
• Improper implementation of authentication and authorization mechanisms
• Injections

It’s worth noting that injection-related vulnerabilities were relatively rare before AI agents took off (previously, they mostly affected web apps). Recently, though, these security issues have become relevant again.

Looking back at a year and a half of the AI boom, one conclusion stands out regarding registered vulnerabilities: AI tool developers are more focused on expanding functionality than on security. This is worth keeping in mind when using these tools. Let’s look at the projects and applications that either integrated AI tools or offered them as the core product. Below is a list of the those with the highest number of registered vulnerabilities for 2025–2026.

TOP AI/LLM-related projects by number of published vulnerabilities, 2025–2026 (download)

Notable vulnerabilities

This section highlights the most significant vulnerabilities published in Q2 2026 that have publicly available descriptions. Since the above already covers several significant vulnerabilities published during the reporting period, this section consists mainly of LLM/AI tool vulnerabilities.

CVE-2026-25253: a gatewayUrl vulnerability in OpenClaw

The issue stems from the fact that the OpenClaw user interface trusts the value of the gatewayUrl parameter passed in the URL and automatically establishes a WebSocket connection to the specified address. During this connection process, it sends an authentication token without any additional user confirmation.

The attack algorithm exploiting this vulnerability works as follows:

1. The application obtains a critical connection address from an external source (the gatewayUrl URL parameter), which is controlled by the attacker.
2. There is no validation before use.
3. The client automatically initiates a connection to the address specified in the parameter, which belongs to the attacker.
4. While connected, the application sends credentials (an access token) to the specified address.

If the attacker obtains a valid token, the consequences depend on that token’s level of access within the system. In general, this could lead to:

• User session compromise
• Execution of operations on the user’s behalf
• Modification of the AI agent configuration
• Unauthorized access to tools and resources connected to the agent
• Under certain OpenClaw configurations, further compromise of the host running the agent

It’s worth noting that the risk of exploitation arises from a combination of several factors: the automatic connection and token transmission, the lack of address trust verification, and the high privileges granted to the local AI agent.

CVE-2026-41948: a path traversal vulnerability in the Dify AI platform

The vulnerability lets an authenticated user craft a request that enables the application to escape its permitted tenant and gain access to internal REST APIs that weren’t meant for that user. The root cause is insufficient normalization and validation of the URL path before it’s passed to the internal service.

Depending on the Dify configuration, the consequences can include:

• Unauthorized access to internal service interfaces
• Breach of isolation between workspaces
• Exposure of internal service information
• Conditions favorable to further attacks when combined with other vulnerabilities

The use of Dify in enterprise AI platforms is particularly risky, since internal services there tend to hold elevated privileges.

CVE-2026-45386: an improper access control vulnerability in Open WebUI

In Open WebUI, pin/unpin operations on messages are write operations, since they modify that message’s metadata (is_pinned, pinned_by, pinned_at). In vulnerable versions, however, before performing these actions, the API only checked for read access to the channel (a chat between a user or group and the AI) containing the message, not permission to modify its content. As a result, a user with a role limited to viewing messages could still change a message’s pinned status.

The vulnerability’s mechanism works as follows:

1. The user initiates an action that changes the state of an object.
2. The application treats this action as a regular read request.
3. Only channel view permission is checked.
4. The application performs a write without verifying the required user authorization.

This violates one of the fundamental principles of access control models — namely, that any operation that changes the state of data must be checked for the appropriate write or moderation permissions, regardless of whether the object itself is readable.

Although the vulnerability doesn’t lead to arbitrary code execution or compromise of sensitive data, it can affect data integrity and collaborative workflows. Potential consequences of exploitation include unauthorized pinning or unpinning of messages, disruption of channel moderators’ and administrators’ activities, changes to the display order of important information, and even the potential spread of false or misleading information by altering the channel containing a pinned message.

Open WebUI is widely used as an interface for interacting with local and enterprise LLMs. In these systems, pinned messages often contain important instructions, announcements, or tips for users. The ability to modify them with minimal privileges can disrupt collaborative workflows, cause confusion, and undermine trust in information published by administrators and moderators.

CVE-2026-45501: a vulnerability in Microsoft Exchange

The vulnerability stems from improper neutralization of user input when generating Exchange web pages. As a result, the browser may interpret specially crafted data as active content instead of plain text.

Although Microsoft categorizes the potential impact of exploiting this vulnerability as spoofing, flaws like this can lead to alteration of displayed content, imitation of trusted interfaces, actions on behalf of the user within an active session, and abuse of user trust.

It’s worth noting that issues like this are still relevant in modern software, given that mechanisms like Content Security Policy and various parsers were specifically created to help developers neutralize dangerous parts of user page content.

Conclusion and advice

Q2 brought the first significant results of AI automation adoption in software development and vulnerability hunting tools. This research shows that beyond traditional patch management, organizations now need real-time monitoring of systems and access controls, since infrastructure and everyday applications now contain far more AI functionality that could lead to compromise.

Accordingly, besides quickly detecting infrastructure vulnerabilities and managing security patches, modern enterprise-grade security solutions need to provide a broad range of preventive measures for tracking the overall health of systems and workstations. Kaspersky Next meets these requirements by combining proactive mechanisms with the ability to respond promptly to emerging threats.

OpenClaw, which was previously known as Clawdbot and Moltbot, is today one of the most successful and fast‑growing ecosystems for AI agents, recognized worldwide. The project quickly became popular with users because of its flexibility and ability to solve fairly complex tasks that previously required a lot of time for automation and execution. A dedicated marketplace appeared quickly after the project started gaining traction, where developers and users began publishing tools that integrate with OpenClaw. Currently, employees all over the world use OpenClaw to automate their tasks, often unaware of risks this practice introduces to them and their employers.

In this article we will examine several security aspects of OpenClaw, look at how attackers can target this system, which vulnerabilities are already known, and how to protect your organization against these issues.

OpenClaw skills

The project’s success was ensured by the fact that the agent accepts natural language instructions, does not require knowledge of programming languages, and allows the use of skills, which expand its capabilities. The overall architecture of OpenClaw can be seen below:

The OpenClaw overall architecture

As shown in the diagram, the system is designed to be used with agent skills. These skills can reside locally on the system where the agent is installed or they can be obtained from external sources. At the time of writing this article, a dedicated hub named “ClawHub” is used for sharing skills with other users.

One of the key features of OpenClaw skills is that they are easy to create and do not require coding. A skill is in essence a set of commands written in natural language, although it can contain code. Currently, there is a general description of the skill format: it is usually a text file named SKILL.md, although more complex variants may exist. The primary requirement for these files is that they use a plaintext format. To illustrate what this looks like, here is a fragment of a skill:

Openclaw skill example

The applications for OpenClaw skills are quite broad and can include everyday tasks like checking email, performing routine operations and calculations on a computer, as well as more complex pipelines that handle testing, research, or software development. For most actions, the agent requires access to the operating system’s file system, as well as to the tokens and keys of the systems it will interact with. All necessary data are usually provided by users either through environment variables or in plaintext files located alongside the agent.

Since many skills enable automation of work processes, employees worldwide actively use them. This fact, combined with the widespread adoption of the system and the overall popularity of artificial‑intelligence technologies, has attracted attackers to the project.

OpenClaw vulnerabilities

In less than two years, around 530 vulnerabilities have been discovered both in OpenClaw itself and in the underlying technologies. That said, the publication of OpenClaw vulnerabilities in the CVE database began only in February 2026. Below is a breakdown of these vulnerabilities by severity.

As shown in the chart, the number of high-severity vulnerabilities is quite large. Most of these vulnerabilities fundamentally involve issues with storing sensitive data and operating with excessively high privileges. Each of them can be exploited to hijack the agent or inject commands that it will execute.

Malicious skills

Besides exploiting vulnerabilities and deceiving users, there are more specific attack vectors against OpenClaw, namely, skills.

Research logically draws a parallel between supply‑chain attacks and the distribution of malicious skills. However, unlike usual supply-chain attacks, creating malicious skills is trivial because there is no longer a need to develop custom malware. Despite this, until February 7, 2026, no skills had undergone even a basic security check, which allowed malicious skills to appear immediately. Our scan of the skill hub in April identified 24 accounts that were distributing more than 600 malicious skills. Overall, open‑source intelligence indicates that over 1100 malicious accounts have been created since January.

Following the investigations and a lengthy effort to clean the skill repository of malicious entries, it was announced that files would undergo preliminary scanning with VirusTotal (VT) and NVIDIA’s SkillSpector. On the one hand, this is a more responsible approach to publishing skills; on the other, because OpenClaw is primarily an agent that executes a set of instructions, detecting malicious activity moves to a different level. Now it is necessary not only to analyze a file for dangerous commands that should be blocked, but also to examine all possible malicious behaviors that could be triggered by a harmful instruction within a skill. An example of a malicious command in natural language:

Example of a malicious command within a skill action

An example of a malicious command using a part of a bash command:

Malicious command inside a skill

The example in the image and similar malicious skills are detected by Kaspersky products as HEUR:Trojan.ANSI.MalClaw.gen.

In addition, Kaspersky products monitor malicious OpenClaw skill activity on the system. Below are detection statistics from our systems that have identified malicious OpenClaw client behavior. The data for June cover the first half of the month.

As shown in the chart, even despite the measures taken to counter the publication of malicious skills, attacks continue. Therefore, it is important to employ layered protection that isolates the OpenClaw agent from critical data and infrastructure systems. We also recommend checking all skills that enter the organization’s perimeter. For this purpose, Kaspersky Scan Engine is suitable. This solution is designed to protect web applications, proxy servers, network attached storage, and mail gateways. It can be integrated into almost any application, and it is easy to deploy and manage.

Malicious skill detected by Scan Engine

Additionally, monitor network accesses used by the agent. For this purpose, the project already provides a sandboxing subsystem and various wrappers for working with APIs and services. Last but not least, develop a comprehensive AI policy and make sure your employees never use third-party tools that they are not explicitly allowed to use.

Small and medium-sized businesses (SMBs) remain attractive targets for cybercriminals – in both mass cyberattacks and sophisticated campaigns targeting larger enterprises through trusted relationship attacks. At the same time, smaller businesses may lack the robust cybersecurity policies and necessary resources to protect themselves against an evolving threat landscape.

Kaspersky believes that raising awareness can help small and medium-sized enterprises develop an effective protection strategy. Ahead of International SMB Day on June 27, Kaspersky presents the findings of its 2026 threat analysis for SMBs, which includes real-world examples of attacks.

Key findings

• In the first four months of 2026, Kaspersky solutions detected over 33,300 cyberattacks on SMBs masquerading as popular artificial intelligence (AI) tools – almost five times more than in 2025 and 39% more than the number of attacks disguised as the office and collaboration tools that Kaspersky’s research focuses on.
• Popular messengers and communication services remained the attacker’s most widespread lure, with almost 415,000 attacks involving fake messenger apps and video conferencing software.
• The attackers follow trends: the AI tools Claude and OpenClaw (ex-ClawdBot/MoltBot), which have gained popularity in 2026, were among the common AI lures.
• Fraudsters use fake AI tools to scam businesses out of money, while corporate accounts on social media also remain targets.
• The majority of initial accesses to corporate infrastructures sold on the dark web are allegedly accesses to SMBs. This could be because SMBs tend not to be as well protected as large enterprises and, at the same time, may be trusted contractors for those well-protected enterprises.

Malware and potentially unwanted applications (PUAs) disguised as popular services

Kaspersky researchers used data from Kaspersky Security Network (KSN) to explore how frequently malicious and unwanted files are disguised as legitimate applications that may be used by SMBs. KSN is a system for processing anonymized cyberthreat-related data shared voluntarily by Kaspersky users. For this part of the report, only anonymized data received from users of Kaspersky solutions for SMBs were analyzed.

According to a survey by the Small Business & Entrepreneurship Council (SBE Council), small business owners continue to embrace artificial intelligence and digital transformation as they maintain a generally positive outlook on the economy. Threat actors are also aware of the hype surrounding AI and exploit it for their own benefit. In particular, they actively distribute cyberthreats under the guise of popular AI services.

From January to April 2026, Kaspersky solutions detected 33,352 attacks on SMB users in which malware or potentially unwanted applications for PCs were disguised as five popular AI services. This figure represents an increase of almost five times compared to the previous year. This highlights an evolving trend in which threat actors are weaponizing trust in widely used AI platforms and services, especially popular ones like Claude. Kaspersky experts note that it’s important to download apps from official sources and to verify which apps are available for which platforms.

In the first four months of 2026, Kaspersky researchers also identified more than 1,100 unique samples of malware and PUAs detected in the SMB sector that masqueraded as five popular AI applications, representing a 21% increase compared to the same period of 2025. The samples were mainly different types of Trojware (Trojans and Trojan-like malware), including those capable of downloading and running other malware on compromised devices. Trojware disguises itself as harmless files to trick users into installing them. Their functionality may vary depending on the particular type of Trojware. This may include stealing, deleting, blocking, modifying or copying users’ data, as well as other malicious actions. Trojware therefore represents a highly dangerous cyberthreat to entrepreneurs and businesses.

Kaspersky experts also note that the threat landscape is constantly evolving with new lures appearing all the time. For example, in the first four months of 2026, Kaspersky solutions blocked hundreds of attacks in which malware or PUAs for PCs were disguised as OpenClaw (previously known as Clawdbot or Moltbot).

Other lures for SMBs: Fake communication apps and office software

Kaspersky analysts also explored how attackers leverage other legitimate applications as lures to target SMBs. For example, from January to April 2026, Kaspersky solutions blocked 414,736 attacks on SMB users in which malicious software or PUAs for PCs were disguised as the popular communication apps that Kaspersky’s report focuses on. The number of attacks changed marginally compared to the previous year’s figure, indicating that the lure of fake communication apps remains a serious cyberthreat.

Various fake office applications and collaborative platforms also remain among the lures that attackers may exploit to target SMBs. According to Kaspersky telemetry, more than 24,000 attacks were detected from January to April 2026 in which malware or PUAs for PCs were disguised as specific office applications.

In 2026, AI-related baits have become more widespread among cybercriminals than traditional fake office and collaboration tools. Kaspersky experts note that the more publicity and hype there is around certain tools, the more likely a user is to come across a fake package online.

Scammers and phishers tricking victims into providing credentials and funds

In 2026, Kaspersky researchers observed a wide range of phishing campaigns and scams targeting businesses and entrepreneurs. Fraudsters mimic financial and AI services as well as other platforms in order to steal credentials, personal information and funds.

In the following example, fraudsters disguise themselves as a bank that allegedly offers services for businesses (in other similar schemes they may offer business loans). Entrepreneurs are prompted to visit a scam website and enter their data to open a business account. The requested information varies depending on the scam, but may include name, email address, phone number, social security number, date of birth and address. Scammers may then use this data in their schemes or sell it on the dark web.

Kaspersky experts advise: if you encounter such a website, you should not rush to enter any data. First, examine it. Does the purported financial organization actually exist? How old is the website? Check the WHOIS records and read user reviews before entering any information on the page.

Example of a scam page targeting entrepreneurs

As with many other cyberthreats, AI services are also leveraged as a lure in scams. For example, Kaspersky experts identified a scam website for an AI service “built for contractors”. According to the text on the fraudulent page, the tool can help with “estimates, invoices and schedule”. However, in reality, in such schemes victims usually receive nothing after paying for a subscription, while the scammers get all the money.

Example of a scam page promoting an AI tool

Kaspersky experts note that business accounts on social networks and messengers remain attractive targets for cybercriminals in 2026. In one scheme, phishers distributed notifications with fake alerts related to companies’ business pages. The notifications claimed that Facebook’s review system had detected behavior that seriously violated its Community Standards and Advertising Policies. To avoid permanent restriction of their business page on the social network, owners were prompted to fill out an appeal form and provide personal and business email addresses, phone numbers, as well as the name of their business page and the password for their social network account. The attackers’ goal was to obtain credentials. To reduce user vigilance  and appear legitimate, fraudsters also sent victims a fake appeal code.

Example of a fake notification

Email threats: Fake online documents and exploitation of legitimate platforms

Email remains one of the most widely used channels for cyberattacks targeting enterprises, including small and medium-sized businesses. In 2026, attackers have frequently combined email distribution with the exploitation of legitimate third-party platforms. This is how phishers and scammers usually attempt to bypass traditional email filters and exploit user trust in reputable services. Kaspersky researchers have also observed a large number of schemes targeting corporate users in which phishers and scammers use fake online documents or nonexistent meetings as bait.

In one recent scheme detected by Kaspersky, the attackers sent a fake notification disguised as a letter from OneDrive. The victim was prompted to access the document by clicking a button, but in reality, it led to a phishing website where users risked losing their confidential data. To make the email appear legitimate, the attackers added a phrase designed to  lower the victim’s vigilance: “This item is encrypted and hosted within your secure cloud perimeter.” They also parsed the recipient’s email address and used the extracted data in the fake notification text so that the email looked like a standard notification from this type of service: “[email address domain as company name] has successfully uploaded a new file for [the user’s name as stated in their email address].”

Example of a phishing scheme with fake online documents

Attackers also use other pretexts to trick victims into sharing confidential information, for example fake compliance issues. In the example below, the attackers posed as Apple representatives. The fake notification stated: “Apple has identified a compliance issue related to Google Ads campaigns directing traffic to Apple product detail pages associated with the victim’s seller account.” However, the button in the email led to a phishing website where users are tricked into sharing confidential data.

Example of a fake compliance issue notification

Kaspersky experts observed another notable two-stage scheme aimed at stealing credentials from corporate emails, which involved distributing an invitation to a nonexistent meeting. The scheme is deployed in two stages. In stage one, a corporate user receives an email about a fictitious meeting. After clicking the “Accept Meeting Invitation” button, the user is redirected to a legitimate Zoom Docs (previous Zoom canvas brand) page. In stage two, the victim is prompted to click a hyperlink that reads “Click Here to Accept Meeting”. However, the URL of a phishing page is hidden behind this hyperlink.

Example of an email with a fake meeting

Zoom Docs page containing the phishing link

Malware is also actively distributed via email. In 2025, individuals and corporate users encountered over 144 million malicious and potentially unwanted email attachments, representing a 15% increase from the previous year.

Kaspersky experts note that the lures used in subject lines and texts of malicious emails can appear relatively harmless and rather unsophisticated. In the example below, the attackers target businesses with a fake request for “the best quote for the items attached.” However, the attached file actually contains a Trojan.

Example of a malicious email

Corporate infrastructure access for sale: Posts on the dark web

To assess threat actor activity, Kaspersky Digital Footprint Intelligence experts analyzed hundreds of posts offering initial access to corporate infrastructures published on dark web forums from January to April of both 2025 and 2026. Kaspersky experts note that a single post may contain several offers for access to different allegedly compromised companies.

Example of a post on a darknet forum

Initial access brokers (IABs) sell initial access to compromised businesses, for example, via RDP or web shells. In their posts, IABs may provide information about the region where the allegedly compromised companies are located, their industry and revenue, as well as the type of access. IABs sell access that the buyers can then use for different purposes, including ransomware attacks, stealing corporate confidential information or other fraudulent activity. The price of initial access on dark web forums may depend on the revenue, industry or location of the allegedly compromised companies, or on the access privileges. For example, accounts with admin rights are usually more expensive because they can provide attackers with a wide range of possibilities.

According to the research, there were more posts offering initial access to companies of different sizes located in the Middle East (up 53% from last year), Africa (up 40%) and Latin America (up 17%). Meanwhile the number of posts related to companies located in Europe decreased by 34%. According to Kaspersky experts, this decline can be partially explained by the closure of a dark web forum containing such posts around the time of the study. The number of publications related to companies located in the APAC region also decreased slightly (down 4%), but remained at a consistently significant level for the second year in a row.
At the same time, the number of posts where the region was not specified decreased by 56% in 2026 compared to the previous year. Kaspersky analysts assume that this may indicate that initial access posts from IABs are becoming more targeted and unique.

Share of posts with initial access offers by business size

For this research, Kaspersky experts defined a small business as having an annual revenue of up to US$50 million, and a medium-sized business as having an annual revenue of between US$50 million and US$1 billion.

According to Kaspersky’s research, at the beginning of 2026 the share of posts on dark web forums with offers of initial access to allegedly compromised small businesses was larger than the shares of posts offering access to medium, large or nonprofit organizations. However, this share decreased in the first four months of 2026 compared to the same period in 2025. The share of posts concerning medium‑sized organizations also remained significant for two consecutive years. Taken together, posts concerning small and medium‑sized organizations account for more than half of all the analyzed posts with initial access offers on dark web forums.

At the same time for a certain number of posts initial access brokers didn’t indicate companies’ revenue, therefore, making it impossible to determine the size of the company.

Kaspersky experts note that despite the prevalence of posts concerning small businesses, threat actors may target medium‑sized businesses because they generate higher revenues than small businesses and may have weaker security defenses than large businesses.

SMBs can also become targets as a part of trusted relationship attacks, which enable the attackers to reach larger organizations. According to the Global Report by Kaspersky Security Services, the share of trusted relationship attacks among the initial vectors increased from 12.7% in 2024 to 15.5% in 2025. Therefore, the common belief that small and medium‑sized enterprises are of no interest to attackers is a misconception. Companies of all sizes need to understand the cyberthreat landscape, adhere to cybersecurity rules, implement appropriate cybersecurity solutions, and continuously improve employee awareness.

Cybersecurity action plan for SMBs

SMBs can reduce risks and ensure business continuity by investing in comprehensive cybersecurity solutions and increasing employee awareness. To protect themselves from the ever-evolving threat landscape, companies are advised to follow these rules:

1. Define access rules for corporate resources such as internet services, email accounts, shared folders, and online documents. Keep access lists up to date and revoke access promptly when employees leave the company.
2. Regularly back up important data to ensure the preservation of corporate information in case of emergencies.
3. Establish clear guidelines for using external services and resources. Create well-defined procedures for coordinating specific tasks, such as implementing new software, with the IT department and other responsible managers. Develop short, easy-to-understand cybersecurity guidelines for employees, with a special focus on account and password management, email protection, and safe web browsing. A well-rounded training program will equip employees with the necessary knowledge and ability to apply it in practice.
4. Raise employees’ security awareness. Conduct dedicated training to teach staff how to detect and address potential threats, and track their educational progress. Organizations can achieve this with the Kaspersky Automated Security Awareness Platform through interactive online modules and simulated phishing campaigns that build sustainable cyber hygiene habits across all teams.
5. Implement specialized cybersecurity solutions that fit your budget, size, and industry requirements, with an emphasis on scalability and ease of integration.
   1. Kaspersky Small Office Security Premium is an easy-to-use solution that protects against advanced threats and also provides access to security awareness training for employees, making it ideal for micro-businesses.
   2. Small and medium-sized enterprises with more mature IT expertise should consider Kaspersky Next Optimum, which is designed specifically for growing organizations and offers real-time protection, threat visibility, as well as EDR and XDR investigation and response capabilities.
6. Protect your business against email-borne threats. Kaspersky Security for Mail Server, a comprehensive email security platform that offers robust, multi-layered protection at mailbox and gateway levels, can help with this. Powered by machine learning and leading global threat intelligence, it effectively addresses all mail security challenges.
7. Adopt specialized solutions such as Kaspersky Digital Footprint Intelligence to monitor the surface, deep, and dark webs for information about a company’s credentials, leaked data, and lookalike websites. Small and medium-sized companies with limited IT security budgets can partner with a managed security service provider (MSSP) to access this comprehensive digital risk protection service at an affordable, subscription-based price point.

Executive Summary: Bypassing Boundaries in Enterprise AI Infrastructure

The massive global adoption of artificial intelligence (AI) and large language models (LLMs) has fundamentally rewritten the enterprise threat landscape. Modern high-compute bare metal clusters, cloud nodes, and developer workstations now routinely host dense data science stacks running frameworks such as TensorRT LLM, PyTorch, and deep CUDA execution layers. To process complex matrix calculations efficiently, low-privilege user mode tasks require direct pipelines straight to the underlying graphics acceleration hardware.

By Yarden Porat

AI agents need memory. Frameworks like LangGraph provide it through checkpointers – persistence layers that store execution state. But what happens when that persistence layer isn’t locked down?

Key Points

• Check Point Research analyzed LangGraph, an open-source framework for stateful AI agents with over 50 million monthly downloads, and uncovered three vulnerabilities in its persistence layer.
  
• Two of them chain into remote code execution: a SQL injection in the SQLite checkpointer (CVE-2025-67644) and an unsafe msgpack deserialization (CVE-2026-28277).

• A third, parallel issue (CVE-2026-27022) introduces the same injection class into the Redis checkpointer.
  
• Who’s at risk: teams self-hosting LangGraph with the SQLite or Redis checkpointer, where the application exposes get_state_history() with a user-controlled filter. LangChain’s managed cloud service, LangSmith Deployment (formerly LangGraph Platform), runs PostgreSQL and is not vulnerable.

• LangChain patched all three issues. Users should update to langgraph-checkpoint-sqlite 3.0.1+, langgraph 1.0.10+, and langgraph-checkpoint-redis 1.0.2+.

Background

LangGraph is an open-source framework for building stateful, multi-agent AI systems with built-in persistence. It’s an extension of LangChain, with over 50 million monthly downloads according to PyPI stats.

Checkpointers are LangGraph’s persistence layer that stores execution state at each step. LangGraph supports two checkpointer implementations: SQLite and PostgreSQL.

Vulnerability #1: SQL Injection (CVE-2025-67644)

The SQLite Checkpointer Database Schema:
The SQLite checkpointer uses an internal table called checkpoints with the following structure:

CREATE TABLE checkpoints (
    thread_id TEXT NOT NULL,
    checkpoint_ns TEXT NOT NULL DEFAULT '',
    checkpoint_id TEXT NOT NULL,
    parent_checkpoint_id TEXT,
    type TEXT,
    checkpoint BLOB,
    metadata BLOB,
    PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
);

The metadata column stores additional contextual information about each checkpoint in JSON format. For example:

{
  "user_id": "alice",
  "step": 1,
  "source": "input"
}

The list() Function and Filtering:

When calling the list() function on sqliteSaver (the checkpointer), the filter parameter is used to query checkpoints based on their metadata:

def list(
    self,
    config: RunnableConfig | None,
    *,
    filter: dict[str, Any] | None = None,  # Used to filter by metadata
    before: RunnableConfig | None = None,
    limit: int | None = None,
) -> Iterator[CheckpointTuple]:

The filter parameter is passed to an internal function called _metadata_predicate, which constructs the SQL WHERE clause to query checkpoints by their metadata fields.

# process metadata query
    for query_key, query_value in filter.items():
        operator, param_value = _where_value(query_value)
        predicates.append(
            f"json_extract(CAST(metadata AS TEXT), '$.{query_key}') {operator}"
        )
        param_values.append(param_value)

    return (predicates, param_values)

The Injection

The vulnerability exists in how _metadata_predicate handles the query_key from the filter dictionary.
Notice this critical line:

f"json_extract(CAST(metadata AS TEXT), '$.{query_key}') {operator}"

An attacker-controlled filter could provide a query_key with a ' character that will escape the JSON path string and inject arbitrary SQL code.

Injection -> Arbitrary Deserialization

To understand how SQL injection leads to arbitrary deserialization, we need to see the complete picture.
Here’s the SQL query that gets executed in list():

query = f"""SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
FROM checkpoints
{where}
ORDER BY checkpoint_id DESC"""

This query retrieves checkpoint data from the database, including the checkpoint’s BLOB column.
The results are then processed:

async for (
    thread_id,
    checkpoint_ns,
    checkpoint_id,
    parent_checkpoint_id,
    type,
    checkpoint,  # ← This comes directly from the SQL query results
    metadata,
) in cur:  # ← cur contains the query results
    # ... 
    yield CheckpointTuple(
        # ...
        self.serde.loads_typed((type, checkpoint)),  # ← Deserialization
        # ...
    )

The checkpoint contains serialized data, and when fetched gets deserialized.

The Attack

Using SQL injection in the WHERE clause, an attacker can inject a UNION SELECT that adds their own row to the query results:

SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
FROM checkpoints
WHERE ... (injected: ') UNION SELECT 'thread1', 'ns', 'checkpoint1', NULL, 'msgpack', X'', '{}' -- )
ORDER BY checkpoint_id DESC

The injected UNION SELECT returns a fake checkpoint row where the checkpoint column contains attacker-controlled serialized data. When the code loops through the query results, it deserializes this malicious checkpoint’s BLOB, giving the attacker arbitrary deserialization

Vulnerability #2: MsgPack Unsafe Deserialization (CVE-2026-28277)

Now let’s examine what happens during deserialization. The self.serde.loads_typed() function that deserializes checkpoint data looks like this:

def loads_typed(self, data: tuple[str, bytes]) -> Any:
    type_, data_ = data
    if type_ == "null":
        return None
    elif type_ == "bytes":
        return data_
    elif type_ == "bytearray":
        return bytearray(data_)
    elif type_ == "json":
        return json.loads(data_, object_hook=self._reviver)
    elif type_ == "msgpack":
        return ormsgpack.unpackb(
            data_, ext_hook=self._unpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
        )
    elif self.pickle_fallback and type_ == "pickle":
        return pickle.loads(data_)
    else:
        raise NotImplementedError(f"Unknown serialization type: {type_}")

Formats

1. Pickle –  is disabled by default
2. JSON –  The json.loads() with object_hook was discussed in our LangGrinch research, but does not lead to code execution
3. Msgpack – This is the one we are interested in

What is msgpack?

MessagePack (msgpack) is a binary serialization format designed to be faster and more compact than JSON. LangGraph uses ormsgpack, a Rust-based implementation with Python bindings.

Msgpack Extensions

MessagePack allows developers to define custom extension types to handle additional data types beyond its built-in primitives. LangGraph implemented its own extension handler to support serialization of custom Python objects.

When the type_ is msgpack, the code calls:

ormsgpack.unpackb(data_, ext_hook=self._unpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS)
```
The `ext_hook` parameter points to LangGraph's custom implementation: `_msgpack_ext_hook`.

```python
def _msgpack_ext_hook(code: int, data: bytes) -> Any:
    if code == EXT_CONSTRUCTOR_SINGLE_ARG:
        try:
            tup = ormsgpack.unpackb(
                data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
            )
            # module, name, arg
            return getattr(importlib.import_module(tup[0]), tup[1])(tup[2])
        except Exception:
            return

When an attacker controls the serialized data, they control both the extension code and the data bytes.

The vulnerability

If we pass a msgpack with EXT_CONSTRUCTOR_SINGLE_ARG code, and the tuple:

1. os
2. system
3. Command (“echo PWN > /tmp/pwned.txt” for example)

When this line executes:

return getattr(importlib.import_module(tup[0]), tup[1])(tup[2])

It will:

1. Import the os module

2. Get the system function from it

3. Call os.system("echo PWN > /tmp/pwned.txt")

This gives an attacker arbitrary code execution – by calling os.system() with attacker-controlled commands, they can execute any shell command on the server.

The Attack Chain: Combining Both Vulnerabilities

Now let’s walk through how an attacker chains these two vulnerabilities together to achieve remote code execution.

The Entry Point: When a developer exposes get_state_history(), it internally calls the checkpointer’s list() method to retrieve historical checkpoints:

def get_state_history(
    self,
    config: RunnableConfig,
    *,
    filter: Optional[Dict[str, Any]] = None,
    before: Optional[RunnableConfig] = None,
    limit: Optional[int] = None,
) -> Iterator[StateSnapshot]:
    # ...
    for checkpoint_tuple in self.checkpointer.list(config, filter=filter, before=before, limit=limit):
        # Process and return checkpoint data

If the filter parameter comes from user input without sanitization, an attacker controls the dictionary keys passed to the SQL injection vulnerability.

The Attack Flow

1. Craft Malicious Payload: The attacker prepares a msgpack payload containing instructions to execute arbitrary code (e.g., run a shell command).

2. Exploit SQL Injection: The attacker sends a malicious filter parameter that exploits the SQL injection vulnerability. This injection adds a fake checkpoint row to the database query results, where the checkpoint column contains their malicious msgpack payload.

3. Trigger Deserialization: When the application processes the query results, it encounters the injected fake checkpoint and deserializes the malicious msgpack data.

4. Code Execution: The unsafe deserialization executes the attacker’s payload, giving them remote code execution on the server.

Vulnerability #3: SQL Injection in the Redis Checkpointer (CVE-2026-27022)

The same injection class affects langgraph-checkpoint-redis: user-controlled keys in the filter dictionary are interpolated directly into the query instead of bound as parameters. Preconditions match CVE-2025-67644 (the application exposes get_state_history() with a user-controlled filter and uses the Redis checkpointer). Patched in langgraph-checkpoint-redis 1.0.2.

Additional SQL Injection Findings

Beyond the primary SQL injection in the filter parameter, we identified additional defense-in-depth SQL injection issues in both the SQLite and PostgreSQL checkpointers. These involved direct concatenation of integer values (such as LIMIT and ttl parameters) into SQL queries instead of using parameterized bindings.

Since Python doesn’t enforce type hints at runtime, these parameters could still accept malicious string input. We worked with the LangChain team during disclosure to remediate these issues using parameterized queries.

Disclosure Timeline

2025-11-19: CVE-2025-67644 (SQL injection), CVE-2026-28227 (msgpack deserialization) And CVE-2026-27022 (Redis injection) disclosed to LangChain team

2025-12-10: CVE-2025-67644 fixed and publicly released in langgraph-checkpoint-sqlite 3.0.1

2026-02-20: CVE-2026-27022  fixed and publicly released in langgraph-checkpoint-redis 1.0.2

2026-03-05: CVE-2026-28277  fixed and publicly released in langgraph-checkpoint 4.0.1

Note on Vendor Response

The LangChain team responded quickly to fix the critical SQL injection vulnerability, which effectively breaks the attack chain described in this research. They continue to work methodically on additional remediation efforts, including the msgpack deserialization issue.

Additional Research

There was significant community research into LangGraph security during November and December 2025. Other security researchers independently discovered CVE-2025-67644 and CVE-2026-28277. Full credits can be found in LangChain’s security advisories.

The post From SQLi to RCE – Exploiting LangGraph’s Checkpointer appeared first on Check Point Research.

Introduction

Containerization using Docker has become firmly established in modern development standards, significantly increasing the speed and convenience of deploying various services. Developers often use ready-made Docker images, making only minimal changes. The largest repository of container images is the Docker Hub service.

Container-hosted infrastructure is an attractive target for attackers. At a minimum, a compromised container can be used for DDoS attacks, cryptocurrency mining, or traffic proxying. The list of threats does not end there: once an attacker gains control of a container, they can steal or destroy data directly from it, access neighboring containers, or even attempt to escape the container, compromising the entire enterprise network.

At the same time, the infrastructure inside containers is typically updated less frequently and may contain outdated and vulnerable software versions. When deploying third-party images or modifying them for a specific environment, it is easy to make configuration errors that attackers can later exploit. And due to the architectural characteristics of containers, developers often face constraints when preparing images; to overcome these, they may resort to insecure solutions they find online.

In other words, containerized infrastructure can be both the simplest and the most lucrative target to exploit. Therefore, its security requires heightened attention. To minimize the risk of successful attacks on container infrastructure, it is essential to check the final Docker images, including all underlying layers, for vulnerabilities and misconfigurations. The easiest way to do this is by analyzing the Dockerfile; however, it is not always available for inspection. Moreover, it typically defines how to build layers on top of a base image from an external repository whose reliability cannot be guaranteed.

Image analysis results in Kaspersky Container Security

To help users identify insecure configurations and potential vulnerabilities within them, we have added our AI assistant to Kaspersky Container Security.KIRA (the assistant’s name) uses artificial intelligence to analyze the image and identify potential issues within, along with recommendations on how to fix them.

As part of this study, we asked KIRA to analyze a number of popular community images, and later in this article, we’ll show you the results.

Software vulnerabilities and compromise of update sources

One of the key security issues with using pre-built images is that developers do not update them in a timely manner. A Docker image is, by its very nature, a snapshot of a specific Linux distribution after packages have been installed on it. However, in most cases, it does not receive security updates on its own, unlike traditional Linux servers, where these updates are automatically installed by specialized services, such as unattended-upgrades in Debian-based distributions and dnf-automatic in RedHat-based distributions.

To apply updates to a Docker image, it must be rebuilt and redeployed. Often, this process is not automated, and some updates require additional effort to verify their correct operation, modify configurations when upgrading to new software versions, and so on. As a result, many popular images do not receive timely updates, which significantly increases the risks associated with their use.

An image that was secure at build time accumulates vulnerabilities as they are discovered in the packages installed within it, which over time significantly increases the opportunities for a successful attack on the container.

Vulnerable versions of web applications and network services accessible from the internet immediately become targets of various malicious campaigns. For example, just one day after the discovery of the CVE-2025-55182 vulnerability in React Server Components, our honeypots recorded numerous attack attempts related to this vulnerability. It was adopted by operators of many malicious campaigns, ranging from classic cryptocurrency miners to variants of Mirai and Gafgyt. Attackers are constantly adding new distribution methods and can use dozens of exploits targeting various vulnerabilities and configuration errors in popular services. Often, the same vulnerabilities are used in self-propagation mechanisms from already compromised hosts. For example, in a malicious campaign to spread the Dero miner, attackers use infected containers to automatically search for and infect new targets.

In addition to vulnerabilities that can be exploited remotely, attackers are rapidly adding local vulnerabilities to their arsenal, used to gain root privileges and escape the container: in the Kinsing malware campaign, attackers used CVE-2023-4911 (Looney Tunables) to elevate privileges, and in the perfctl campaign, the CVE-2021-4034 (PwnKit) vulnerability was used for the same purpose. The access gained was used to install a rootkit that hides the presence of perfctl on the system.

To assess the situation with unpatched vulnerabilities in containers, we took a random sample of 100 images, which included various popular solutions with 10,000 to 1 million downloads on DockerHub. In the 64 images we scanned, we found outdated software versions with critical vulnerabilities. For example, some images contained the CVE-2025-49844 vulnerability in the Redis server, leading to RCE by leveraging a vulnerability in the Lua parser; the current CVE-2026-24061 vulnerability in nginx, which in some configurations leads to a server process crash, and with ASLR disabled, again, to RCE; vulnerabilities CVE-2025-32463 in sudo and CVE-2023-4911 in glibc, allowing an attacker to gain root privileges with local access. At the same time, only one in ten Docker images from the analyzed sample is fully up to date.

TOP 10 Critical Vulnerabilities with PoC/Exploits available as shown in the Kaspersky Container Security Dashboard

It is worth noting that, of course, not every discovered vulnerability can be directly exploited by attackers. A practical risk arises when the vulnerable application or library is actually in use, and the conditions necessary for exploitation – which vary significantly from vulnerability to vulnerability – are met. Nevertheless, updates must not be ignored, as the risk of vulnerabilities being exploited – both individually and in various combinations – cannot be predicted in each specific case, and even vulnerabilities that seem harmless at first glance can ultimately pose a serious risk of compromise.

A record number of vulnerabilities in a single image

However, frequent updates have a downside. Every rebuild that downloads new packages from source repositories introduces an additional risk of a supply chain attack – a compromised dependency or a modified base image could silently inject malicious code into your environment precisely through an update. During our analysis of images from the sample, we did not find any signs of supply chain attacks. However, in March 2026, a supply chain incident occurred in the Trivy and LiteLLM projects. In the case of Trivy, the infected file was injected directly into the container image in the official repositories.

Detecting potentially malicious software using one of the images as an example

This leads to a difficult choice: infrequent updates leave known vulnerabilities unpatched within the image, while frequent updates increase the risk of supply chain compromise. Therefore, to protect your infrastructure, you need not only to regularly update base images but also to take a more comprehensive approach, specifically by pinning dependencies to known-good versions and scanning the resulting images for malware upon update.

Configuration vulnerabilities

Even a container with a fully updated image can be compromised if it is configured incorrectly. Embedding keys and secrets in the image, disabling authentication in network services, default passwords, and insecure file access permissions – all of these can be exploited by attackers in one way or another to achieve their goals.

Insecure image configurations detected by KCS based on rules

The situation is exacerbated by the fact that errors may be introduced by the authors of the original image, which complicates their detection, as this requires analyzing every layer and the command that generated it. As with vulnerabilities, not every configuration error leads to compromise: it all depends on the container’s role, its network accessibility, and many other factors. But the very use of insecure settings will sooner or later lead to errors appearing in images where their consequences will be significantly more dangerous.

Standard rules are often insufficient for analyzing problematic configurations. To gain a deeper understanding of the context and assess potential risks, AI tools can be used. Later in this section, we will examine examples of typical insecure configurations we discovered while scanning public images from Docker Hub, along with the descriptions of issues and risk mitigation methods provided by the KIRA AI assistant.

Example of container analysis using KIRA

Insecure handling of credentials

Use of default passwords

In some cases, containers may use default passwords set via environment variables or directly in Dockerfile. If these passwords are not overridden, attackers will be able to access the application by using the default password.

RUN |1 DEBIAN_FRONTEND=noninteractive /bin/sh -c echo [removed]:[removed] | chpasswd

According to KIRA’s analysis, the user’s password is stored in plain text in the image layer history. Anyone who gains access to the image – whether through a public registry, a compromised build environment, or other means – will be able to extract the password. If SSH or another form of interactive access is enabled in the container, this could lead to its complete compromise and allow attackers to move laterally within the infrastructure.

Passwords may be present in environment variables. Consider the following Dockerfile snippet:

ENV SERVERNAME=localhost WWW_PATH_CONF=/etc/apache2/apache2.conf WWW_PATH_ROOT=/var/www HTTPS=on PKP_CLI_INSTALL=0 PKP_DB_HOST=db PKP_DB_NAME=pkp PKP_DB_USER=pkp PKP_DB_PASSWORD=changeMePlease PKP_WEB_CONF=/etc/apache2/conf-enabled/pkp.conf PKP_CONF=config.inc.php PKP_CMD=/usr/local/bin/pkp-start

In this example, the environment variable PKP_DB_PASSWORD is set to changeMePlease. If the user forgets to override it, the application will use the password that can be obtained from Dockerfile.

Let’s look at another image:

/bin/sh -c #(nop)  ENV MOODLE_URL=<a href="http://0.0.0.0/">http://0.0.0.0</a> MOODLE_ADMIN admin       MOODLE_ADMIN_PASSWORD [removed]      MOODLE_ADMIN_EMAIL admin@example.com MOODLE_DB_HOST     MOODLE_DB_PASSWORD       MOODLE_DB_USER     MOODLE_DB_NAME    MOODLE_DB_PORT 3306

For this image, Dockerfile specifies that the administrator password is hardcoded in the ENV directive and remains in the image metadata (layer history, docker inspect). Anyone who gains access to the image (registry, build cache) will be able to extract this secret and compromise the account.

To eliminate these risks, ensure that no passwords are specified in Dockerfile. If authentication is required, you can use orchestrator mechanisms (secrets) or generate a temporary password when starting the container via the entrypoint script, without saving it in the layers. We also recommend using mechanisms for securely passing secrets at runtime (Docker secrets, Kubernetes Secrets) or, as a last resort, passing them via --secret during the build with BuildKit, but under no circumstances should they be left in the final image.

Passing passwords via command arguments

In some cases, passwords may be exposed when passed via command-line arguments, as these arguments are visible to all users on the system:

/bin/sh -c #(nop)  HEALTHCHECK &{[""CMD-SHELL"" ""mysql --protocol TCP -u\""root\"" -p\""$MYSQL_ROOT_PASSWORD\"" -e \""SELECT 1;\""""] ""15s"" ""30s"" ""0s"" '\x05'}

In the example provided, the MySQL superuser password is passed into the healthcheck command in plaintext, making it visible when viewing the process list (ps aux), in audit logs, and in monitoring systems. If the attacker gains read access to the container’s processes or logs, they can extract the password and gain full control of the database.

To fix this issue, the healthcheck should use a local connection via a Unix socket with default authentication (if the auth_socket plugin is configured for root), or create a dedicated user with minimal privileges (e.g., only USAGE), without a password or with a password passed via a secure file (--defaults-file with restricted permissions). You can also use the MYSQL_PWD environment variable for healthcheck authentication, but it remains visible in /proc.

Privilege escalation in the container

One of the most common vectors for initial compromise of Linux systems is RCE in web applications and network services. Typically, these services have minimal privileges, which complicates attackers’ subsequent actions: dumping credentials, covering their tracks, attempting to escape the container, and much more.

The situation worsens significantly if the attacker gains root privileges, as this allows them to fully control all processes within the container, conceal their activity, and use methods to escape the container. For example, they can compromise the host if the container is privileged, a Docker socket is mounted inside it, or other insecure configurations and vulnerabilities exist that cannot be exploited with standard user privileges.

Similarly, this simplifies network attacks on neighboring containers, the orchestrator, and various internal services, making this configuration error a potential link in the chain for compromising the entire network.

Attacks on sudo

One of the simplest privilege escalation methods is executing arbitrary commands as root using sudo without entering a password. Consider the following example:

/bin/sh -c set -xe;     apt-get update &&       apt-get -y install sudo;       echo ""solr ALL=(ALL) NOPASSWD: ALL"" >/etc/sudoers.d/solr;

Analyzing this configuration using KIRA immediately highlights the main issue: by installing the sudo package and setting NOPASSWD: ALL for the solr, the user severely violates the principle of least privilege. The Solr platform does not require such broad privileges to run within a container; instead, they create an easy path for escalating to root.

echo 'postgres ALL=(ALL:ALL) NOPASSWD:ALL' >> /etc/sudoers

In another example of an insecure configuration, NOPASSWD:ALL privileges are granted to a PostgreSQL database user, which is a direct and severe weakening of the access control policy. If an attacker gains the ability to execute code on behalf of the postgres user – through a vulnerability in a network service, an SQL injection, or by compromising of one of the processes – they will immediately and unconditionally be able to execute any commands on behalf of the root user. This is equivalent to the entire container running as root.

As a risk mitigation measure, we recommend completely removing this directive. The minimum necessary commands requiring privileges should be delegated on a case-by-case basis via sudoers with explicit specification of allowed executables and parameters, using NOPASSWD only as a last resort and for specific utilities.

Our AI assistant KIRA can identify even more complex insecure configurations, such as allowing passwordless sudo for the entire sudo group — by modifying existing rules.

perl -i -pe 's/\bALL$/NOPASSWD:ALL/g' /etc/sudoers

The risk in this example is that the command replaces standard declarations requiring authentication with passwordless execution of all commands for any user within the sudo group – potentially including postgres, should it be assigned to that group. This expands the attack surface to all group members, turning each of them into a potential point for instant privilege escalation.

To mitigate the risks, we recommend not modifying the global sudoers policy, keeping the standard password requirement, or using a more secure escalation mechanism – such as gosu to run a specific process on behalf of another user without permanent privileges.

Insecure file permissions

Another common vector for privilege escalation is insecurely configured file and directory permissions. Most often, for convenience, container image authors use 777 permissions, which allow anyone – including unprivileged users – to freely create and delete files, as well as modify their contents. This can lead to both privilege escalation and the ability for an unprivileged attacker to delete or modify logs, among other undesirable consequences.

Consider the following command:

chmod 0777 /usr/share/cargo /usr/share/cargo/bin

The risk is that directories containing binary files and scripts will become writable by any container user. This allows a low-privileged attacker to replace utilities included in cargo or add new malicious executables. When these tools are subsequently invoked, especially as the root user or via sudo, the attacker’s code will execute with the inherited privileges of the calling process, leading directly to a local privilege escalation.

To mitigate the risks, you can set the minimum necessary permissions: chmod 0755 for directories and chmod 0755/0644 for the corresponding files. The owner should be root, and only the owner should be allowed to write. Do not use chmod 777 on any system paths.

Lack of integrity checks

Downloading software without verifying its integrity can make the infrastructure vulnerable to software tampering.

For example, this risk may arise when downloading a distribution via HTTP:

RUN /bin/sh -c wget -qO- ""<a href="http://acestream.org/downloads/linux/acestream_3.1.49_debian_9.9_x86_64.tar.gz">http://acestream.org/downloads/linux/acestream_3.1.49_debian_9.9_x86_64.tar.gz</a>"" | tar --extract --gzip -C /opt/acestream

Using HTTP without verifying the archive’s integrity creates conditions for a man-in-the-middle attack during the image build phase. An attacker controlling the communication channel or DNS can replace the archive with malicious content, which will compromise the container and the entire environment in which it runs.

To mitigate the risks, you can configure connections to web resources to use HTTPS only — if the resource supports this protocol. You can also download the archive without extracting it, compare its checksum (SHA256) with the checksum from a trusted source, and only then extract it. It is advisable to store the verified archive in an internal artifact repository to avoid direct downloads from the network.

There will still be a MitM risk even if certificate verification is disabled:

wget --no-check-certificate<a href="https://github.com/phpvirtualbox/phpvirtualbox/archive/refs/heads/7.2-dev.zip"> https://github.com/phpvirtualbox/phpvirtualbox/archive/refs/heads/7.2-dev.zip</a> -O phpvirtualbox.zip

The absence of TLS certificate verification allows an attacker controlling the network segment to replace the downloaded ZIP archive with malicious content. Since the archive contains PHP code that will be executed by the web server, compromise during the build phase will result in the deployment of a backdoor or data leakage.

To mitigate the risks, remove the --no-check-certificate flag; after downloading, calculate the SHA256 hash of the archive and verify it against a known reference value (the release page or a local repository of trusted hashes). Additionally, consider using a fixed release (tag) rather than the floating 7.2-dev branch.

Conclusion

Docker containers have become a very popular means of deploying software, and attackers are by no means oblivious to this trend. They are rapidly adding software vulnerabilities and configuration errors to their arsenal and carrying out attacks on supply chains. They can compromise container infrastructure for a wide variety of purposes, from cryptocurrency mining to encrypting data for ransom or stealing information critical to the company.

Our research found that 64 out of 100 container images for popular applications contain critically vulnerable software, and only 10% are fully up to date. We also identified numerous insecure configurations, including passwords stored in plaintext in Dockerfiles and excessive privileges granted to users and processes.

To detect and prevent these threats, it is essential to strictly adhere to security measures: audit image configurations, securely manage secrets used in images, apply security updates in a timely manner, scan their contents for malware with every update, and follow industry-standard best practices for enhancing security.

This approach requires specialized solutions built to accommodate the unique characteristics of container environments. Kaspersky Container Security ensures the security of containerized applications at every stage of their lifecycle, from development to operation. The product protects an organization’s business processes, helps ensure compliance with industry standards and security regulations, and enables the implementation of secure software development practices.

Artificial intelligence (AI) is changing the shape of the application attack surface. A traditional application assessment usually starts with familiar questions, such as:

Artificial intelligence (AI) is now squarely on the frontlines of information security. However, as is often the case when the pace of technological innovation is very rapid, security often ends up being a secondary consideration. This is increasingly evident from the ad-hoc nature of many implementations, where organizations lack a clear strategy for responsible AI use.

Attack surfaces aren’t just expanding due to risks and vulnerabilities in AI models themselves but also in the underlying infrastructure that supports them. Many foundation models, as well as the data sets used to train them, are open-source and readily available to developers and adversaries alike.

Unique risks to AI models

According to Ruben Boonen, CNE Capability Development Lead at IBM: “One problem is that you have these models hosted on giant open-source data stores. You don’t know who created them or how they were modified, and there are a number of issues that can occur here. For example, let’s say you use PyTorch to load a model hosted on one of these data stores, but it has been changed in a way that’s undesirable. It can be very hard to tell because the model might behave normally in 99% of cases.”

Recently, researchers discovered thousands of malicious files hosted on Hugging Face, one of the largest repositories for open-source generative AI models and training data sets. These included around a hundred malicious models capable of injecting malicious code onto users’ machines. In one case, hackers set up a fake profile masquerading as genetic testing startup 23AndMe to deceive users into downloading a compromised model capable of stealing AWS passwords. It was downloaded thousands of times before finally being reported and removed.

In another recent case, red team researchers discovered vulnerabilities in ChatGPT’s API, in which a single HTTP request elicited two responses indicating an unusual code path that could theoretically be exploited if not addressed. This, in turn, could lead to data leakage, denial of service attacks and even escalation of privileges. The team also discovered vulnerabilities in plugins for ChatGPT, potentially resulting in account takeover.

While open-source licensing and cloud computing are key drivers of innovation in the AI space, they’re also a source of risk. On top of these AI-specific risk areas, general infrastructure security concerns also apply, such as vulnerabilities in cloud configurations or poor monitoring and logging processes.

AI models are the new frontier of intellectual property theft

Imagine pouring huge amounts of financial and human resources into building a proprietary AI model, only to have it stolen or reverse-engineered. Unfortunately, model theft is a growing problem, not least because AI models often contain sensitive information and can potentially reveal an organization’s secrets should they end up in the wrong hands.

One of the most common mechanisms for model theft is model extraction, whereby attackers access and exploit models through API vulnerabilities. This can potentially grant them access to black-box models — like ChatGPT — at which point they can strategically query the model to collect enough data to reverse engineer it.

In most cases, AI systems run on cloud architecture rather than local machines. After all, the cloud provides the scalable data storage and processing power required to run AI models easily and accessibly. However, that accessibility also increases the attack surface, allowing adversaries to exploit vulnerabilities like misconfigurations in access permissions.

“When companies provide these models, there are usually client-facing applications delivering services to end users, such as an AI chatbot. If there’s an API that tells it which model to use, attackers could attempt to exploit it to access an unreleased model,” says Boonen.

Red teams keep AI models secure

Protecting against model theft and reverse engineering requires a multifaceted approach that combines conventional security measures like secure containerization practices and access controls, as well as offensive security measures.

The latter is where red teaming comes in. Red teams can proactively address several aspects of AI model theft, such as:

• API attacks: By systematically querying black-box models in the same way adversaries would, red teams can identify vulnerabilities like suboptimal rate limiting or insufficient response filtering.
• Side-channel attacks: Red teams can also carry out side-channel analyses, in which they monitor metrics like CPU and memory usage in an attempt to glean information about the model size, architecture or parameters.
• Container and orchestration attacks: By assessing containerized AI dependencies like frameworks, libraries, models and applications, red teams can identify orchestration vulnerabilities, such as misconfigured permissions and unauthorized container access.
• Supply chain attacks: Red teams can probe entire AI supply chains spanning multiple dependencies hosted in different environments to ensure that only trusted components like plugins and third-party integrations are being used.

A thorough red teaming strategy can simulate the full scope of real-world attacks against AI infrastructure to reveal gaps in security and incident response plans that could lead to model theft.

Mitigating the problem of excessive agency in AI systems

Most AI systems have a degree of autonomy with regard to how they interface with different systems and respond to prompts. After all, that’s what makes them useful. However, if systems have too much autonomy, functionality or permissions — a concept OWASP calls “excessive agency” — they can end up triggering harmful or unpredictable outputs and processes or leaving gaps in security.

Boonen warns that components, such as optical character recognition (OCR) for PDF files and images which multimodal systems rely on to process inputs, “can introduce vulnerabilities if they’re not properly secured”.

Granting an AI system excessive agency also expands the attack surface unnecessarily, thus giving adversaries more potential entry points. Typically, AI systems designed for enterprise use are integrated into much broader environments spanning multiple infrastructures, plugins, data sources and APIs. Excessive agency is what happens when these integrations result in an unacceptable trade-off between security and functionality.

Let’s consider an example where an AI-powered personal assistant has direct access to an individual’s Microsoft Teams meeting recordings stored in OneDrive for Business, the purpose being to summarize content in those meetings in a readily accessible written format. However, let’s imagine that the plugin doesn’t only have the ability to read meeting recordings but also everything else stored in the user’s OneDrive account, in which many confidential information assets are also stored. Perhaps the plugin even has write capabilities, in which case a security flaw could potentially grant attackers an easy pathway for uploading malicious content.

Once again, red teaming can help identify flaws in AI integrations, especially in environments where many different plugins and APIs are in use. Their simulated attacks and comprehensive analyses will be able to identify vulnerabilities and inconsistencies in access permissions, as well as cases where access rights are unnecessarily lax. Even if they don’t identify any security vulnerabilities, they will still be able to provide insight into how to reduce the attack surface.

The post How red teaming helps safeguard the infrastructure behind AI models appeared first on Security Intelligence.

Artificial intelligence (AI) and machine learning (ML) have entered the enterprise environment.

According to the IBM AI in Action 2024 Report, two broad groups are onboarding AI: Leaders and learners. Leaders are seeing quantifiable results, with two-thirds reporting 25% (or greater) boosts to revenue growth. Learners, meanwhile, say they’re following an AI roadmap (72%), but just 40% say their C-suite fully understands the value of AI investment.

One thing they have in common? Challenges with data security. Despite their success with AI and ML, security remains the top concern. Here’s why.

Full steam ahead: How AI and ML get smarter

Historically, computers did what they were told. Thinking outside the box wasn’t an option — lines of code dictated what was possible and permissible.

AI and ML models take a different approach. Instead of rigid structures, AI and ML models are given general guidelines. Companies supply vast amounts of training data that help these models “learn,” in turn improving their output.

A simple example is an AI tool designed to identify images of dogs. The underlying ML structures provide basic guidance — dogs have four legs, two ears, a tail and fur. Thousands of images of both dogs and not-dogs are provided to AI. The more pictures it “sees,” the better it becomes at differentiating dogs.

Off the rails: The risks of unauthorized model modification

If attackers can gain access to AI models, they can modify model outputs. Consider the example above. Malicious actors compromise business networks and flood training models with unlabeled images of cats and images incorrectly labeled as dogs. Over time, model accuracy suffers and outputs are no longer reliable.

Forbes highlights a recent competition that saw hackers trying to “jailbreak” popular AI models and trick them into producing inaccurate or harmful content. The rise of generative tools makes this kind of protection a priority — in 2023, researchers discovered that by simply adding strings of random symbols to the end of queries, they could convince generative AI (gen AI) tools to provide answers that bypassed model safety filters.

And this concern isn’t just conceptual. As noted by The Hacker News, an attack technique known as “Sleepy Pickle” poses significant risks for ML models. By inserting a malicious payload into pickle files — used to serialize Python object structures — attackers can change how models weigh and compare data and alter model outputs. This could allow them to generate misinformation that causes harm to users, steal user data or generate content that contains malicious links.

Staying the course: Three components for better security

To reduce the risk of compromised AI and ML, three components are critical:

1) Securing the data

Accurate, timely and reliable data underpins usable model outputs. The process of centralizing and correlating this data, however, creates a tempting target for attackers. If they can infiltrate large-scale AI data storage, they can manipulate model outputs.

As a result, enterprises need solutions that automatically and continuously monitor AI infrastructure for signs of compromise.

2) Securing the model

Changes to AI and ML models can lead to outputs that look legitimate but have been modified by attackers. At best, these outputs inconvenience customers and slow down business processes. At worst, they could negatively impact both reputation and revenue.

To reduce the risk of model manipulation, organizations need tools capable of identifying security vulnerabilities and detecting misconfigurations.

3) Securing the usage

Who’s using models? With what data? And for what purpose? Even if data and models are secured, use by malicious actors may put companies at risk. Continuous compliance monitoring is critical to ensure legitimate use.

Making the most of models

AI and ML tools can help enterprises discover data insights and drive increased revenue. If compromised, however, models can be used to deliver inaccurate outputs or deploy malicious code.

With Guardium AI security, businesses are better equipped to manage the security risks of sensitive models. See how.

The post The straight and narrow — How to keep ML and AI training on track appeared first on Security Intelligence.

Cybersecurity requires creativity and thinking outside the box. It’s why more organizations are looking at people with soft skills and coming from outside the tech industry to address the cyber skills gap. As the threat landscape becomes more complex and nation-state actors launch innovative cyberattacks against critical infrastructure, there is a need for cybersecurity professionals who can anticipate these attacks and develop creative preventive solutions.

Of course, a lot of cybersecurity work is mundane and repetitive — monitoring logs, sniffing out false positive alerts, etc. Artificial intelligence (AI) has been a boon in filling the talent gaps when it comes to these types of tasks. But AI has also proven useful for many of the same things that creative thought brings to the threat table, such as addressing more sophisticated threat actors, the rapid increase of data and the hybrid infrastructure.

However, many companies are seeing the value of AI, especially generative AI (gen AI), in handling a greater share of creative work — not just in cybersecurity but also in areas like marketing and public relations, writing and research. But are these organizations using AI in a way that could threaten the importance of human creativity in threat detection?

Why creativity is important to cybersecurity

The very simple reason why cybersecurity requires innovative people is that threat actors are already coming up with novel approaches to how to get into your system. Are they using gen AI to launch their attacks? You bet they are; phishing emails have never been more grammatically constructed or realistic. But before AI was available, threat actors were designing social engineering attacks that attracted clicks. Now, they have advanced beyond “how can we lure in victims” to “how can we get more out of a single attack after we lure in the victims.”

Creativity isn’t just coming up with new ideas. It is also the ability to see things through a big-picture lens and discern historical data or where to find information you might not know you need to look for. For example, creative thought is required for the following security tasks:

• Threat hunting or predicting a threat actor’s move or finding their tracks in a system
• Finding buried evidence in a forensic search
• Understanding historical data in anomaly detection
• Ability to tell a real email or document versus a well-designed phishing attack
• Verifying new zero day attacks and other malware variants found in otherwise unknown vulnerabilities

AI can augment human creativity, but gen AI gets a lot of things wrong. Users have found themselves in situations where AI claimed plagiarism on original work or AI hallucinations offered false information that nullified the research of human analysts. AI algorithms are also susceptible to bias that could lead to false positives.

AI’s role in creative cybersecurity and beyond

While many creative people, cybersecurity professionals and beyond, see gen AI as a mixed blessing, many embrace the technology because it is a huge timesaver.

“Gen AI can help prototype much faster because the large language models can take over the refactoring and documentation of code,” wrote Aili McConnon in an IBM blog post. Also, the article pointed out, AI tools can help users create prototypes or visualize their ideas in minutes versus hours or days.

Creativity married to AI can help identify future leaders. According to research from IBM, two-thirds of company leaders found that AI is driving their growth, with four specific use cases — IT operations, user experience, virtual assistants and cybersecurity — most commonly favored by leaders.

“A Learner will typically copy predefined scenarios using out-of-the-box technologies,” Dr. Stephan Bloehdorn, Executive Partner and Practice Leader, AI, Analytics and Automation-IBM Consulting DACH, was quoted in the study. “But a Leader develops custom innovations.”

Over-reliance on AI?

As gen AI becomes more ubiquitous in the workplace and as more creative folks and leaders rely on it as a way to put their ideas in motion, are we also relying on the technology to the point that it could lead to a degradation of other important necessary skills, like the ability to analyze data and create viable solutions?

It is unclear if organizations are over-relying on gen AI, according to Stephen Kowski, Field CTO at SlashNext Email Security+, but it is becoming more of a designed feature due to unintended consequences related to resource allocation in organizations.

“While AI excels at processing massive volumes of threat data, real-world attacks constantly evolve beyond historical patterns, requiring human expertise to identify and respond to zero-day threats,” said Kowski in an email interview. “The key is achieving the right balance where AI handles high-volume routine detection while skilled analysts investigate novel attack patterns and determine strategic responses.”

Yet, Kris Bondi, CEO and Co-Founder of Mimoto, isn’t worried about AI leading to a degradation of skills — at least not for the foreseeable future.

“One of the biggest challenges for cybersecurity professionals is having too many alerts and too many false positives. AI is only able to automate a small percentage of responses. It’s more likely that AI will eventually automate additional requirements for someone deemed to be suspicious or the elevation of alert so that a human can analyze the situation,” Bondi said via email.

However, organizations should watch out for AI’s role in defining threat-hunting parameters. “If AI is the sole driver defining threat hunting parameters without spot-checks or audits, the threat intelligence approach could eventually be focused in the wrong area. The answer is more reliance on critical thinking and analytical skills,” said Bondi.

Embracing creativity in an AI-driven world

AI overall, and gen AI in particular, are going to be part of the business world going forward. It is going to play a vital role in how organizations and analysts approach cybersecurity defenses and mitigations. But the soft skills that creative thought depends on will still play an important and necessary role in cybersecurity.

“Rather than diminishing soft skills, AI integration has the opportunity to elevate the importance of communication, collaboration and strategic thinking, as security teams must effectively convey complex findings to stakeholders,” said Kowski. “The human elements of cybersecurity — leadership, adaptability and cross-functional partnership — become even more critical as AI handles the technical heavy lifting.”

The post Will AI threaten the role of human creativity in cyber threat detection? appeared first on Security Intelligence.

Human communication is multimodal. We receive information in many different ways, allowing our brains to see the world from various angles and turn these different “modes” of information into a consolidated picture of reality.

We’ve now reached the point where artificial intelligence (AI) can do the same, at least to a degree. Much like our brains, multimodal AI applications process different types — or modalities — of data. For example, OpenAI’s ChatGPT 4.0 can reason across text, vision and audio, granting it greater contextual awareness and more humanlike interaction.

However, while these applications are clearly valuable in a business environment that’s laser-focused on efficiency and adaptability, their inherent complexity also introduces some unique risks.

According to Ruben Boonen, CNE Capability Development Lead at IBM: “Attacks against multimodal AI systems are mostly about getting them to create malicious outcomes in end-user applications or bypass content moderation systems. Now imagine these systems in a high-risk environment, such as a computer vision model in a self-driving car. If you could fool a car into thinking it shouldn’t stop even though it should, that could be catastrophic.”

Multimodal AI risks: An example in finance

Here’s another possible real-world scenario:

An investment banking firm uses a multimodal AI application to inform its trading decisions, processing both textual and visual data. The system uses a sentiment analysis tool to analyze text data, such as earnings reports, analyst insights and news feeds, to determine how market participants feel about specific financial assets. Then, it conducts a technical analysis of visual data, such as stock charts and trend analysis graphs, to offer insights into stock performance.

An adversary, a fraudulent hedge fund manager, then targets vulnerabilities in the system to manipulate trading decisions. In this case, the attacker launches a data poisoning attack by flooding online news sources with fabricated stories about specific markets and financial assets. Next, they launch an adversarial attack by making pixel-level manipulations — known as perturbations — to stock performance charts that are imperceptible to the human eye but enough to exploit the AI’s visual analysis abilities.

The result? Due to the manipulated input data and false signals, the system recommends buying orders at artificially inflated stock prices. Unaware of the exploit, the company follows the AI’s recommendations, while the attacker, holding shares in the target assets, sells them for an ill-gotten profit.

Getting there before adversaries

Now, let’s imagine that the attack wasn’t really carried out by a fraudulent hedge fund manager but was instead a simulated attack by a red team specialist with the goal of discovering the vulnerability before a real-world adversary could.

By simulating these complex, multifaceted attacks in safe, sandboxed environments, red teams can reveal potential vulnerabilities that traditional security systems are almost certain to miss. This proactive approach is essential for fortifying multimodal AI applications before they end up in a production environment.

According to the IBM Institute of Business Value, 96% of executives agree that the adoption of generative AI will increase the chances of a security breach in their organizations within the next three years. The rapid proliferation of multimodal AI models will only be a force multiplier of that problem, hence the growing importance of AI-specialized red teaming. These specialists can proactively address the unique risk that comes with multimodal AI: cross-modal attacks.

Cross-modal attacks: Manipulating inputs to generate malicious outputs

A cross-modal attack involves inputting malicious data in one modality to produce malicious output in another. These can take the form of data poisoning attacks during the model training and development phase or adversarial attacks, which occur during the inference phase once the model has already been deployed.

“When you have multimodal systems, they’re obviously taking input, and there’s going to be some kind of parser that reads that input. For example, if you upload a PDF file or an image, there’s an image-parsing or OCR library that extracts data from it. However, those types of libraries have had issues,” says Boonen.

Cross-modal data poisoning attacks are arguably the most severe since a major vulnerability could necessitate the entire model being retrained on an updated data set. Generative AI uses encoders to transform input data into embeddings — numerical representations of the data that encode relationships and meanings. Multimodal systems use different encoders for each type of data, such as text, image, audio and video. On top of that, they use multimodal encoders to integrate and align data of different types.

In a cross-modal data poisoning attack, an adversary with access to training data and systems could manipulate input data to make encoders generate malicious embeddings. For example, they might deliberately add incorrect or misleading text captions to images so that the encoder misclassifies them, resulting in an undesirable output. In cases where the correct classification of data is crucial, as it is in AI systems used for medical diagnoses or autonomous vehicles, this can have dire consequences.

Red teaming is essential for simulating such scenarios before they can have real-world impact. “Let’s say you have an image classifier in a multimodal AI application,” says Boonen. “There are tools that you can use to generate images and have the classifier give you a score. Now, let’s imagine that a red team targets the scoring mechanism to gradually get it to classify an image incorrectly. For images, we don’t necessarily know how the classifier determines what each element of the image is, so you keep modifying it, such as by adding noise. Eventually, the classifier stops producing accurate results.”

Vulnerabilities in real-time machine learning models

Many multimodal models have real-time machine learning capabilities, learning continuously from new data, as is the case in the scenario we explored earlier. This is an example of a cross-modal adversarial attack. In these cases, an adversary could bombard an AI application that’s already in production with manipulated data to trick the system into misclassifying inputs. This can, of course, happen unintentionally, too, hence why it’s sometimes said that generative AI is getting “dumber.”

In any case, the result is that models that are trained and/or retrained by bad data inevitably end up degrading over time — a concept known as AI model drift. Multimodal AI systems only exacerbate this problem due to the added risk of inconsistencies between different data types. That’s why red teaming is essential for detecting vulnerabilities in the way different modalities interact with one another, both during the training and inference phases.

Red teams can also detect vulnerabilities in security protocols and how they’re applied across modalities. Different types of data require different security protocols, but they must be aligned to prevent gaps from forming. Consider, for example, an authentication system that lets users verify themselves either with voice or facial recognition. Let’s imagine that the voice verification element lacks sufficient anti-spoofing measures. Chances are, the attacker will target the less secure modality.

Multimodal AI systems used in surveillance and access control systems are also subject to data synchronization risks. Such a system might use video and audio data to detect suspicious activity in real-time by matching lip movements captured on video to a spoken passphrase or name. If an attacker were to tamper with the feeds, resulting in a slight delay between the two, they could mislead the system using pre-recorded video or audio to gain unauthorized access.

Getting started with multimodal AI red teaming

While it’s admittedly still early days for attacks targeting multimodal AI applications, it always pays to take a proactive stance.

As next-generation AI applications become deeply ingrained in routine business workflows and even security systems themselves, red teaming doesn’t just bring peace of mind — it can uncover vulnerabilities that will almost certainly go unnoticed by conventional, reactive security systems.

Multimodal AI applications present a new frontier for red teaming, and organizations need their expertise to ensure they learn about the vulnerabilities before their adversaries do.

The post Stress-testing multimodal AI applications is a new frontier for red teams appeared first on Security Intelligence.

The rising influence of artificial intelligence (AI) has many organizations scrambling to address the new cybersecurity and data privacy concerns created by the technology, especially as AI is used in cloud systems. Apple addresses AI’s security and privacy issues head-on with its Private Cloud Compute (PCC) system.

Apple seems to have solved the problem of offering cloud services without undermining user privacy or adding additional layers of insecurity. It had to do so, as Apple needed to create a cloud infrastructure on which to run generative AI (genAI) models that need more processing power than its devices could supply while also protecting user privacy, stated a ComputerWorld article.

Apple is opening the PCC system to security researchers to “learn more about PCC and perform their own independent verification of our claims,” the company announced. In addition, Apple is also expanding its Apple Security Bounty.

What does this mean for AI security going forward? Security Intelligence spoke with Ruben Boonen, CNE Capability Development Lead at IBM, to learn what researchers think about PCC and Apple’s approach.

SI: ComputerWorld reported this story, saying that Apple hopes that “the energy of the entire infosec community will combine to help build a moat to protect the future of AI.” What do you think of this move?

Boonen: I read the ComputerWorld article and reviewed Apple’s own statements about their private cloud. I think what Apple has done here is good. I think it goes beyond what other cloud providers do because Apple is providing an insight into some of the internal components they use and are basically telling the security community, you can have a look at this and see if it is secure or not.

Also good from the perspective that AI is constantly getting bigger as an industry. Bringing generative AI components into regular consumer devices and getting people to trust their data with AI services is a really good step.

SI: What do you see as the pros of Apple’s approach to securing AI in the cloud?

Boonen: Other cloud providers do provide high-security guarantees for data that’s stored on their cloud. Many businesses, including IBM, trust their corporate data to these cloud providers. But a lot of times, the processes to secure data aren’t visible to their customers; they don’t explain exactly what they do. The biggest difference here is that Apple is providing this transparent environment for users to test that plane.

SI: What are some of the downsides?

Boonen: Currently, the most capable AI models are very big, and that makes them very useful. But when we want AI on consumer devices, there’s a tendency for vendors to ship small models that can’t answer all questions, so it relies on the larger models in the cloud. That comes with additional risk. But I think it is inevitable that the whole industry will be moving to that cloud model for AI. Apple is implementing this now because they want to give consumers trust to the AI process.

SI: Apple’s system doesn’t play well with other systems and products. How will Apple’s efforts to secure AI in the cloud benefit other systems?

Boonen: They are providing a design template that other providers like Microsoft, Google and Amazon can then replicate. I think it is mostly effective as an example for other providers to say maybe we should implement something similar and provide similar testing capabilities for our customers. So I don’t think this directly impacts other providers except to push them to be more transparent in their processes.

It’s also important to mention Apple’s Bug Bounty as they invite researchers in to look at their system. Apple has a history of not doing very well with security, and there have been cases in the past where they’ve refused to pay out bounties for issues found by the security community. So I’m not sure they’re doing this entirely out of the interest of attracting researchers, but also in part of convincing their customers that they are doing things securely.

That being said, having read their design documentation, which is extensive, I think they’re doing a pretty good job in addressing security around AI in the cloud.

The post Cybersecurity awareness: Apple’s cloud-based AI security system appeared first on Security Intelligence.

Have you ever wished you had an assistant at your security operations centers (SOCs) — especially one who never calls in sick, has a bad day or takes a long lunch? Your wish may come true soon. Not surprisingly, AI-driven SOC “co-pilots” are topping the lists for cybersecurity predictions in 2025, which often describe these tools as game-changers.

“AI-driven SOC co-pilots will make a significant impact in 2025, helping security teams prioritize threats and turn overwhelming amounts of data into actionable intelligence,” says Brian Linder, Cybersecurity Evangelist at Check Point. “It’s a game-changer for SOC efficiency.”

What is an AI-driven SOC co-pilot?

AI-driven SOC co-pilots are generative AI tools that use machine learning to help security analysts run and manage the SOC. Common co-pilot tasks include detecting threats, managing incidents, triaging alerts, predicting new trends and patterns for attacks and breaches and automating responses to threats. Co-pilots may be proprietary tools built by the company for their specific needs or commercially available cybersecurity co-pilots such as Microsoft Copilot.

For example, a co-pilot can review alerts and use AI to predict which are most likely to be a high priority. This reduces a common issue in SOCs: false positives. The analysts can then focus on the alerts that are most likely to be a real threat. Because they are not chasing down noncritical alerts, analysts have more time to spend on actual threats and are more likely to be successful in containing the threat.

Co-pilots can take many different forms in a SOC. Analysts can use the co-pilot similarly to how many people use ChatGPT, assigning it a specific task such as incident response. The analyst enters information about a specific incident, and the co-pilot analyzes data to suggest possible causes as well as how the organizations should respond to the incident. However, you can also use co-pilots to automate parts of the workflow without human intervention, such as monitoring current firewalls and detecting vulnerabilities.

Benefits of using AI-driven SOC co-pilots

Businesses that turn to AI-driven co-pilots to help manage their SOC see a wide range of benefits. Common benefits include:

• Improved productivity: Because it can process a much higher volume of data than even the most efficient cybersecurity analyst, a co-pilot gets significantly more work done in less time. With humans and machines working together, co-pilots are able to more effectively monitor the SOC with fewer human resources.
• Additional time for cybersecurity professionals to complete high-level tasks: When co-pilots handle manual and repetitive tasks, analysts have more time for higher-level tasks such as strategy and analytics. Analysts are more likely to be fully engaged when their day is filled with more interesting work, which reduces burnout.
• Fewer errors: Humans make mistakes, especially with manual tasks such as reviewing logs. While AI tools are only as “smart” as the algorithm and the training data used for the algorithm, they are often able to spot patterns that may be undetectable to humans. This reduces errors and prevents issues that can lead to a breach or attack.
• Quicker response to threats: Whereas humans may not recognize an area of vulnerability or may be slower to respond, a co-pilot uses automation to respond and send a notification immediately. Co-pilots also don’t take bathroom or lunch breaks; they are always “at their desk,” leading to faster response times.
• Reduced impact of worker shortage and skills gaps: When cybersecurity positions are not filled or the analyst does not have the right skills for the job, the company’s risk increases. AI-driven co-pilots can help reduce open positions by taking on various manual tasks, which means greater coverage by the SOC.

Will AI-driven SOC co-pilots replace humans?

Like many AI tools, co-pilots can take over many manual and repetitive tasks currently done by humans. However, the fear of AI replacing the need for humans in the SOC is not likely to become reality. Setting up co-pilots to operate without human oversight or intervention would likely be a mistake. But businesses that have analysts and co-pilots work together can see a reduction in risk, better responses and higher employee satisfaction.

While co-pilots can be the first line of defense in the SOC, companies should set up gen AI tools so that humans remain the ultimate decision-makers. For example, an analyst may set up an automation with an AI-driven co-pilot to monitor and prioritize alerts based on set criteria. Yet, as threat actors begin using new tactics, the analyst may need to change the criteria to catch the latest threats. Once the co-pilot identifies a high-priority alert, the human can ask the tool to analyze the situation and provide recommended next steps. The analyst then uses human judgment to make the best decisions in the situation and instructs the tool to take the next action, such as shutting down systems or taking the network temporarily offline.

Putting AI-driven co-pilots into action in the SOC

When it comes to putting co-pilots in action, consider starting on a small scale with a limited use case. Many organizations use a commercial product to start, leaving open the option to create a proprietary tool in the future. Creating a list of time-consuming tasks in the SOC, especially those that are error-prone or frustrating for analysts, will help you determine which use case to start with. After launching the tool, a single analyst can gather feedback and make changes.

Upon seeing success, your team can begin expanding the use of co-pilots to additional analysts and use cases. By taking a measured approach to using co-pilots and continuously soliciting feedback from the analysts, businesses can create a partnership between analysts and co-pilots that improves human job satisfaction while also keeping the organization more secure.

The post How AI-driven SOC co-pilots will change security center operations appeared first on Security Intelligence.

“A computer can never be held accountable, therefore a computer must never make a management decision.”

– IBM Training Manual, 1979

Artificial intelligence (AI) adoption is on the rise. According to the IBM Global AI Adoption Index 2023, 42% of enterprises have actively deployed AI, and 40% are experimenting with the technology. Of those using or exploring AI, 59% have accelerated their investments and rollouts over the past two years. The result is an uptick in AI decision-making that leverages intelligent tools to arrive at (supposedly) accurate answers.

Rapid adoption, however, raises a question: Who’s responsible if AI makes a poor choice? Does the fault lie with IT teams? Executives? AI model builders? Device manufacturers?

In this piece, we’ll explore the evolving world of AI and reexamine the quote above in the context of current use cases: Do companies still need a human in the loop, or can AI make the call?

Getting it right: Where AI is improving business outcomes

Guy Pearce, principal consultant at DEGI and member of the ISACA working trends group, has been involved with AI for more than three decades. “First, it was symbolic,” he says, “and now it’s statistical. It’s algorithms and models that allow data processing and improve business performance over time.”

Data from IBM’s recent AI in Action report shows the impact of this shift. Two-thirds of leaders say that AI has driven more than a 25% improvement in revenue growth rates, and 72% say that the C-suite is fully aligned with IT leadership about what comes next on the path to AI maturity.

With confidence in AI growing, enterprises are implementing intelligent tools to improve business outcomes. For example, wealth management firm Consult Venture Partners deployed AIda AI, a conversational digital AI concierge that uses IBM watsonx assistant technology to answer potential clients’ questions without the need for human agents.

The results speak for themselves: Alda AI answered 92% of queries correctly, 47% of queries led to webinar registrations and 39% of inquiries turned into leads.

Missing the mark: What happens if AI makes mistakes?

92% is an impressive achievement for Alda AI. The caveat? It was still wrong 8% of the time. So, what happens when AI makes mistakes?

For Pearce, it depends on the stakes.

He uses the example of a financial firm leveraging AI to evaluate credit scores and issue loans. The outcomes of these decisions are relatively low stakes. In the best-case scenario, AI approves loans that are paid back on time and in full. In the worst case, borrowers default, and companies need to pursue legal action. While inconvenient, the negative outcomes are far outweighed by the potential positives.

“When it comes to high stakes,” says Pearce, “look at the medical industry. Let’s say we use AI to address the problem of wait times. Do we have sufficient data to ensure patients are seen in the right order? What if we get it wrong? The outcome could be death.”

As a result, how AI is used in decision-making depends largely on what it’s making decisions about and how these decisions impact both the company making the decisions and those the decision affects.

In some cases, even the worst-case scenario is a minor inconvenience. In others, the results could cause significant harm. 

Taking the blame: Who’s accountable if AI gets it wrong?

In April 2024, a Tesla operating in “full self-driving” mode struck and killed a motorcyclist. The driver of the vehicle admitted to looking at their phone prior to the crash despite active driver supervision being required.

So who takes the blame? The driver is the obvious choice and was arrested on charges of vehicular homicide.

But this isn’t the only path to accountability. There’s also a case to be made in which Tesla bears some responsibility since the company’s AI algorithm failed to spot the victim. Blame could also be placed on governing bodies such as the National Highway Traffic Safety Administration (NHTSA). Perhaps their testing wasn’t rigorous or complete enough.

One could even argue that the creator(s) of Tesla’s AI could be held liable for letting code that could kill someone go live.

This is the paradox of AI decision-making: Is someone at fault, or is everyone at fault? “If you bring all the stakeholders together who should be accountable, where does that accountability lie?” asks Pearce. “With the C-suite? With the whole team? If you have accountability that’s spread over the entire organization, everyone can’t end up in jail. Ultimately, shared accountability often leads to no accountability.”

Drawing the line: Where does AI end?

So, where do organizations draw the line? Where does AI insight give way to human decision-making?

Three considerations are key: Ethics, risk and trust.

“When it comes to ethical dilemmas,” says Pearce, “AI can’t do it.” This is because intelligent tools naturally seek the most efficient path, not the most ethical. As a result, any decision involving ethical questions or concerns should include human oversight.

Risk, meanwhile, is an AI specialty. “AI is good in risk,” Pearce says. “What statistical models do is give you something called a standard error, which lets you know if what AI is recommending has a high or low potential variability.” This makes AI great for risk-based decisions like those in finance or insurance.

Finally, enterprises need to prioritize trust. “There are declining levels of trust in institutions,” says Pearce. “Many citizens don’t feel confident that the data they share is being used in a trustworthy manner.”

For example, under GDPR, companies need to be transparent about data collection and handling and give citizens a chance to opt-out. To bolster trust in AI use, organizations should clearly communicate how and why they’re using AI and (where possible) allow customers and clients to opt out of AI-driven processes.

Decisions, decisions

Should AI be used for management decisions? Maybe. Will it be used to make some of these decisions? Almost certainly. The draw of AI — its ability to capture, correlate and analyze multiple data sets and deliver new insights — makes it a powerful tool for enterprises to streamline operations and reduce costs.

What’s less clear is how the shift to management-level decision-making will impact accountability. According to Pearce, current conditions create “blurry lines” in this area; legislation hasn’t kept pace with increasing AI usage.

To ensure alignment with ethical principles, reduce the risk of wrong choices and engender stakeholder and customer trust, businesses are best served by keeping humans in the loop. Maybe this means direct approval from staff is required before AI can act. Maybe it means the occasional review and evaluation of AI decision-making outcomes.

Whatever approach enterprises choose, however, the core message remains the same: When it comes to AI-driven decisions, there’s no hard-and-fast line. It’s a moving target, one defined by possible risk, potential reward and probable outcomes.

The post AI decision-making: Where do businesses draw the line? appeared first on Security Intelligence.

With cloud infrastructure and, more recently, artificial intelligence (AI) systems becoming prime targets for attackers, security leaders are laser-focused on defending these high-profile areas. They’re right to do so, too, as cyber criminals turn to new and emerging technologies to launch and scale ever more sophisticated attacks.

However, this heightened attention to emerging threats makes it easy to overlook traditional attack vectors, such as human-driven social engineering and vulnerabilities in physical security.

As adversaries exploit an ever-wider range of potential entry points — both new and old — security leaders must strike a balance to ensure that they’re capable of addressing all risks effectively.

Cyber crime is still a human problem

Despite overwhelming hype, technology is not a panacea. It can’t replace human expertise in every domain, and AI alone can’t match the innately human qualities of intuition and creative thinking. Adversaries know this too, which is why the smarter — and much more dangerous — ones use a blend of human- and technology-powered tactics.

While major technical vulnerabilities tend to make the headlines, the reality is that the weakest link is almost always the human element. Almost all attacks involve a social engineering element, and despite the buzz around generative AI and deepfakes helping scale such attacks, it’s human-to-human interaction where the greatest risks lie.

Synthetic content is now all around us, and people are getting better at telling it apart. Whether we get to the point when that’s no longer the case is a topic for another discussion. But for now, the most dangerous and effective social engineering attacks still depend primarily on human conversations, whether by phone, email or even in person. After all, a seasoned attacker can build trust and forge sham relationships in a way that no AI nor deepfake can match.

Cyber espionage remains a serious threat

Take state-sponsored cyber espionage, for example. Highly trained social engineers are a far cry from the typical rabble of independent cyber crime rackets operating off the dark web, who tend to rely more on scale than targeting specific enterprises and individuals. These attackers may target data systems, but when it comes to their own arsenals, their talents in manipulation and deception are by far their greatest weapons.

Technology still has a long way to go before it can come close to matching the age-old tactics of spycraft.

When facing an attacker who can pose effectively as an internal employee or any other trusted individual, someone relying solely on technology to mitigate the threat stands little chance of protecting themselves. That isn’t a technology failure. It’s a process failure, hence why the human element must always be a key factor in any cybersecurity strategy.

Of course, that’s not to say technology doesn’t have a vital role to play in bolstering your cyber defenses. It most certainly does, not least, because more and more routine threats are being automated or are carried out en-masse by attackers who are less skilled or experienced. The value of technology — especially AI-powered cybersecurity automation — exists primarily in its ability to free up time for security leaders to focus on the threats that technology alone can’t solve.

It’s not all about the cloud, either

The majority of business data is now stored in the cloud, and the percentage continues to rise. Many businesses, especially smaller organizations and startups, exclusively use the cloud for data storage and other IT operations. The rise of AI, given how computationally demanding it is, is further accelerating cloud adoption.

Nonetheless, cloud computing isn’t the best option in all situations. On-premises remains the preferred choice for high-performance workloads that require extremely low latencies. In some cases, on-premises computing is also the cheaper option, and that’s unlikely to change in the near future.

Even though more companies are migrating to the cloud, that doesn’t mean they don’t keep sensitive data on-site. For instance, edge computing, which brings data processing closer to where it’s needed, has become a critical enabler in certain use cases. Examples include smart energy grids, remote monitoring of industrial assets and autonomous vehicles. These include cases where you can’t always rely on internet connectivity.

The smarter and better-funded adversaries aren’t just targeting cloud-hosted infrastructure. They’re also setting their sights on local servers and cyber-physical systems, such as industrial control systems and hardware supply chains. The fact that there’s often minimal collaboration between logistics, production and cybersecurity departments makes these risks all the more serious.

Ransomware remains one of the biggest threats targeting on-premises systems despite the small reduction in attacks over the last year. While cloud systems aren’t inherently immune from ransomware attacks, the vast majority target bare-metal hypervisors and local servers. In one recent case, the Akira ransomware group reverted to its earlier double extortion tactics, experimenting with different code frameworks to target systems running ESXi and Linux.

Botnets are another growing concern as the number of IoT devices continues to soar. Used to launch distributed denial of service (DDoS) attacks spanning thousands of devices, these botnets primarily target unsecured IoT devices, like those that monitor and operate industrial machines and critical infrastructure. One recent report discovered that DDoS attacks against critical infrastructure have increased by 55% in the last four years. These attacks don’t directly involve the exfiltration of sensitive data, but given how they can cause widespread disruption, adversaries may rely on them to draw attention away from more serious threats.

Why physical security is still relevant

As security leaders focus on locking down their cloud-hosted assets, they cannot afford to lose sight of the risks facing their physical infrastructure. Sometimes, the easiest way into the cloud is from within.

Even thin clients and dumb terminals — both widely used in high-security environments like healthcare and finance — can potentially give attackers a foothold in wider systems, including cloud infrastructure and remote data centers. Edward Snowden proved that while working at the National Security Agency when he exfiltrated 20,000 government documents stored on the servers in NSA’s headquarters 5,000 miles away. He did so without using any advanced technology. While that happened way back in 2013, and the NSA has long since updated its physical security protocols, the risk is just as relevant today as it was then.

While most thin clients are now protected by multiple layers of security, including encryption and multifactor authentication, these solutions alone can’t fully protect against physical compromise. If an attacker gains access to a terminal — perhaps by way of social engineering — they may be able to compromise it using unauthorized peripherals or by directly manipulating the device’s firmware. This could give them access to the wider network, potentially allowing for the injection of customized malware that goes undetected by regular security scans.

IoT devices are another leading reason behind the expansion of attack surfaces. They often lack adequate security, also giving attackers a potential entry point into the broader computing infrastructures they’re connected to. The fact that these connected technologies are being rolled out en masse in areas like smart cities, critical infrastructure and transportation networks, greatly magnifies such vulnerabilities.

Ultimately, if an attacker is able to get past your physical safeguards, then these connected systems present far easier pathways to an organization’s so-called “crown jewels” than trying to break through multi-layered cloud defenses.

Cloud data is not always the true target

In other cases, data hosted in the cloud might not be the attacker’s end goal. Many companies, such as those subject to stringent data residency regulations or that require high performance for real-time applications, still store their data on on-premises servers.

Some of these systems are air-gapped, meaning they’re entirely disconnected from any other networks, including the Internet itself. While more secure than any cloud-hosted server, at least in theory, their security can’t be taken for granted. For instance, anyone with physical access to the servers may be able to compromise them, either maliciously or accidentally.

Physical security, such as CCTV and biometric security checkpoints, is as important as ever in such cases. But it’s not just about protecting against intentional physical tampering. Indirect attacks orchestrated by highly skilled social engineers can also dupe unsuspecting employees into taking a desired action — such as lending them a biometric security access card.

These are not the sort of adversaries that usually work by email or use AI to scale their attacks – they’re far likelier to deceive someone in person, a tactic as old as humanity itself. In fact, the attacker could be anyone, such as a disgruntled former employee, a hacker operating in the interests of a rival company or even a rogue state.

Bridging the gap between digital and human security

Technology alone can’t protect an organization from the myriad threats out there, and neither can humans keep up with ever-expanding system logs and security information feeds if they’re relying solely on manual processes.

The reality is that you need both, starting with people and using technology to broaden their capabilities. A layered security strategy should typically start with locking down physical access to any data-bearing system or system that is connected to another.

The next layer of defense is the human one. This revolves heavily around security awareness training. But the reality is that many programs are ineffective, either because they lack practical application, are overly reliant on generic content or focus too much on technical factors that are beyond the target audience’s understanding.

Phishing simulations are often similarly limited in their scope, focusing on common lures like trending news topics, a sense of urgency or even outright threats. However, more sophisticated attackers tend to use subtler ways to elicit a response. This could be something as simple as sending messages about a routine policy update regarding company dress code or remote work guidelines. These topics might seem trivial, but they can pique interest, especially when they concern changes to daily routines and work-life balance. Attackers could then use this to dupe unsuspecting victims into divulging sensitive information via a sham survey.

Like any other security measure, physical systems and awareness training will only ever be effective if they’re tested regularly. That’s where physical red teaming comes in. Whereas red teaming in the context of IT focuses on technical measures like penetration testing, physical red teaming is all about having teams try to gain entry to restricted areas and systems. To do so, they might use a blend of simulated social engineering attacks and technology to hack into physical security systems. By attempting to bypass physical security barriers or impersonate staff, red teams can reveal gaps that might otherwise go unnoticed. That’s what makes them a valuable part of any comprehensive information security program.

The post AI and cloud vulnerabilities aren’t the only threats facing CISOs today appeared first on Security Intelligence.

Many times a day worldwide, a boss asks one of their team members to perform a task during a video call. But is the person assigning tasks actually who they say they are? Or is it a deepfake? Instead of blindly following orders, employees must now ask themselves if they are becoming a victims of fraud.

Earlier this year, a finance worker found themselves talking on a video meeting with someone who looked and sounded just like their CFO. After the meeting was over, they then dutifully followed their boss’s instructions to send $200 million Hong Kong dollars, which equals $25 million.

But it wasn’t actually their boss — just an AI video representation called a deepfake. Later that day, the employee realized their terrible mistake after checking with the corporate offices of their multinational firm. They had been a victim of a deepfake scheme that defrauded the organization out of $25 million.

Businesses are often deepfake targets

The term deepfake refers to AI-created content — video, image, audio or text — that contains false or altered information, such as Taylor Swift promoting cookware and the infamous fake Tom Cruise. Even the recent hurricanes hitting the U.S. led to multiple deepfake images, including fake flooded Disney World photos and heartbreaking AI-generated pictures of people with their pets in floodwaters.

While deepfakes, also referred to as synthetic media, targeted at individuals typically serve to manipulate people, cyber criminals targeting businesses are looking for monetary gain. According to the CISA Contextualizing Deepfake Threats to Organizations information sheet, threats targeting businesses tend to fall into one of three categories: executive impersonation for brand manipulation, impersonation for financial gain or impersonation to gain access.

But the recent incident in Hong Kong wasn’t just one employee making a mistake. Deepfake schemes are becoming increasingly common for businesses. A recent Medus survey found that the majority (53%) of finance professionals have been targeted by attempted deepfake schemes. Even more concerning is the fact that more than 43% admitted to ultimately falling victim to the attack.

Watch Unmask the Deepfake

Are deepfake attacks underreported?

The key word from the Medus research is “admitted.”  And it raises a big question. Do people fail to report being a victim of a deepfake attack because they are embarrassed? The answer is probably.  After the fact, it seems obvious it was a fake to other people. And it’s tough to admit that you fell for an AI-generated image.  But the underreporting only adds to the shame and makes it easier for cyber criminals to get away with it.

Most people assume that they could spot a deepfake. But that’s not the case. The Center for Humans and Machines and CREED found a wide gap between people’s confidence in identifying a deepfake and their actual performance. Because many people overestimate their ability to identify a deepfake, it adds to the shame when someone falls victim, which likely leads to underreporting.

Why people fall for deepfake schemes

The employee who was tricked by the deepfake of the CFO to the tune of $25 million later admitted that when they first got the email supposedly from his CFO, the mention of a secret transaction made them wonder if the email was actually a phishing email. But once he got on the video, they recognized other members of his department in the video and decided it was authentic. However, the employee later learned that the video images of his department members were also deepfakes.

Many people who are victims overlook their concerns, questions and doubts. But what makes people, even those educated on deepfakes, push their concerns to the side and choose to believe an image is real? That’s the $1 million — or $25 million — question that we need to answer to prevent costly and damaging deepfake schemes in the future.

Sage Journals asked the question about who was more likely to fall for deepfakes and didn’t find any pattern around age or gender. However, older individuals may be more vulnerable to the scheme and have a hard time detecting it. Additionally, the researchers found that while awareness is a good starting point, it appears to have limited effectiveness in preventing people from falling for deepfakes.

However, computational neuroscientist Tijl Grootswagers of Western Sydney University likely hit the nail on the head as to the challenge of spotting a deepfake: it’s a brand new skill for each of us. We’ve learned to be skeptical of news stories and bias, but questioning the authenticity of an image we can see goes against our thought processes. Grootswagers told Science Magazine “In our lives, we never have to think about who is a real or a fake person. It’s not a task we’ve been trained on.”

Interestingly, Grootswagers discovered that our brains are better at detection without our intervention. He discovered that when people looked at a picture of a deepfake, the image resulted in a different electrical signal to the brain’s visual cortex than a legitimate image or video. When asked why, he wasn’t quite sure — maybe the signal never reached our consciousness due to interference from other brain regions, or maybe humans don’t recognize the signals that an image is fake because it’s a new task.

This means that each of us must begin to train our brain to consider that any image or video that we view could possibly be a deepfake. By asking this question each and every time we begin to act on content, we may be able to begin detecting our brain signals that are spotting the fakes before we can. And most importantly, if we do fall victim to a deepfake, especially at work, it’s key that each of us reports all instances. Only then can experts and authorities begin to curb the creation and proliferation.

The post Are successful deepfake scams more common than we realize? appeared first on Security Intelligence.