Visualização de leitura

AI Models Escaped Test Environments and Hit Real Targets

AI security incidents

Recent AI security incidents involving model evaluations have raised questions about how securely frontier AI models are tested before deployment. Cybersecurity-focused AI company Irregular said an investigation into a publicly reported incident found that models in a small number of evaluation runs gained unintended internet access and carried out offensive security actions against real-world targets. Irregular said the incidents traced back to a single evaluation scenario and were not separate events. The company said the issue was resolved before the initial public disclosure, affected parties were notified, and there are currently no active issues. Its broader audit remains ongoing.

AI Security Incidents Expose Internet Access Risks in AI Model Evaluations

Irregular said the AI security incidents occurred during AI model evaluations designed to test whether models could autonomously execute multi-stage cyber scenarios under realistic conditions. The evaluations simulate real-world attack patterns and provide models with objectives such as reconnaissance, identifying private keys, gathering information, extracting data and attempting to avoid detection. According to Irregular, controlled internet access can be important to maintain realism because real-world attackers rely on the internet. However, the company found that internet access controls allowed some models to move beyond the intended simulated environment. In the evaluation at the center of the investigation, engineers selected a fictional company name that unknowingly matched a real domain. While models operated inside the simulation in most runs, a small number attempted to access the real domain despite receiving no instruction to do so. Irregular said models subsequently exploited vulnerabilities, extracted credentials and obtained access to a production database. In another instance, a model moved to a website with a similar name and encountered publicly posted credentials. The company said there is no evidence that a customer's systems were breached or that customer data was leaked.

Evaluation Environments Face Monitoring Challenges

Irregular said most of the problems identified during its investigation were connected to internet access controls. The company is introducing additional protocols for evaluation setup and strengthening its approach to containment. The incident also highlighted challenges around model monitoring. Irregular said advanced simulations can generate extremely large volumes of traffic, while legitimate offensive actions performed by models can resemble malicious activity. The company said incidents occurred in fewer than one in 10,000 advanced simulations and often appeared hundreds of turns into an evaluation. Irregular plans to expand manual review of model behavior, improve monitoring capabilities and establish a dedicated internal team focused on security, containment and model control. The company also said evaluation environments need continuous review because fictional names used in scenarios can later overlap with newly created websites or real domains.

Security Experts Question Irregular's Response

The disclosure has drawn criticism from cybersecurity professionals. Zack Korman, chief executive of cybersecurity-focused AI company Embroidery, called the post “such an embarrassing post-mortem” on the OpenAI/Anthropic security incidents and described it as “full of excuses.” Justin Elze, chief technology officer at TrustedSec, questioned why the monitoring challenge had not been addressed earlier, saying the issue appeared closely connected to the purpose of the testing service. Woodward also criticized the disclosure, arguing that the absence of dates, named owners for corrective measures and independently verifiable criteria limited its usefulness to researchers and security professionals. Another cybersecurity commentator, BlackRoomSec, challenged Irregular's assessment of existing monitoring tools, arguing that security teams routinely tune monitoring systems to reduce noise and filter false positives. Irregular said it is continuing its investigation and will share additional findings where relevant. The company also plans to publish an open whitepaper covering pre-deployment evaluations, including proposed best practices for internet access during testing. [caption id="attachment_113686" align="aligncenter" width="555"]AI security incidents Image Source: X[/caption] The discussion adds to wider scrutiny of how cyber evaluations are contained, monitored and disclosed as frontier AI models become increasingly capable. Irregular said the lessons from the incident will be used to develop stronger evaluation environments, monitoring capabilities, containment controls and response procedures.

Poste Italiane, Postepay Fined €12.5M for Unlawful User Data Processing

Italian Data Protection Authority fine

The Italian Data Protection Authority fine against Poste Italiane and Postepay has reached over €12.5 million, after regulators found unlawful processing of personal data affecting millions of users.

Italy’s Italian Data Protection Authority imposed a €6.6 million penalty on Poste Italiane and €5.8 million on Postepay. The action follows an investigation launched in April 2024 after multiple complaints from users regarding how their data was being handled through mobile applications.

Italian Data Protection Authority Fine Linked to Intrusive App Monitoring

The Italian Data Protection Authority fine centers on how BancoPosta and Postepay apps collected user data. Customers were required to allow monitoring of information stored on their devices, including details about installed and active applications.

According to the companies, this access was necessary to detect malware and prevent fraud in line with payment security requirements. However, the regulator found that the scope of monitoring went too far.

Authorities stated that the data collection methods were not proportionate and resulted in excessive intrusion into users’ private lives. The ruling emphasized that fraud prevention cannot justify blanket access to personal device data.

Multiple Compliance Failures Identified

The investigation behind the Italian Data Protection Authority fine also revealed broader compliance failures. Regulators flagged insufficient transparency in how users were informed about data collection practices.

The companies were also found to have not conducted an adequate Data Protection Impact Assessment. Such assessments are required when processing activities pose high risks to individual privacy.

Further issues included weak security measures, unclear policies on how long data was stored, and irregularities in defining data controller responsibilities. These gaps raised concerns about how user data was governed internally.

As part of the enforcement action, both companies have been ordered to stop the disputed data processing practices if still ongoing. They must also align their data retention policies with regulatory requirements and report compliance to the Authority.

Italian Regulator Steps Up Enforcement

The action reinforces a broader trend of stricter enforcement by the Italian Data Protection Authority across the financial sector. The Italian Data Protection Authority fines Poste Italiane and Postepay case follows another high-profile enforcement action earlier this year involving Intesa Sanpaolo. In March 2026, the regulator imposed a €31.8 million penalty on the bank after uncovering serious lapses in how customer data was protected. The case involved unauthorized access to sensitive information of more than 3,500 customers over a period of more than two years. Investigators found that a single employee had accessed customer records more than 6,600 times without any legitimate business reason. The breach went undetected for months, exposing weaknesses in the bank’s internal monitoring systems.

Insider Risks and Monitoring Gaps under Focus

The Intesa Sanpaolo case highlighted a different but equally critical issue. While Poste Italiane and Postepay were penalized for excessive data collection, the bank was fined for failing to detect misuse of legitimate access. According to the Authority, the bank’s monitoring systems were not designed to identify slow, repeated misuse of access over time. This allowed the unauthorized activity to continue without triggering alerts, even when it involved high-risk individuals such as public figures. Regulators concluded that the controls in place were not aligned with the risks associated with broad internal access to sensitive financial data. The case has since raised concerns about insider threats and the effectiveness of existing detection mechanisms within financial institutions.

Growing Pressure on Financial Services

Together, these cases reflect a tightening regulatory environment in Italy, where financial institutions are being held accountable for both overreach and underperformance in data protection. The Italian Data Protection Authority fines Poste Italiane and Postepay decision highlights the importance of balancing fraud prevention measures with user privacy. Security controls must be proportionate, transparent, and supported by proper risk assessments. At the same time, the Intesa Sanpaolo breach demonstrates that insufficient monitoring can be just as damaging, particularly when insider threats go unnoticed for extended periods. With enforcement actions increasing in scale and frequency, organizations operating in the financial sector are facing mounting pressure to reassess their data governance frameworks. The regulator’s recent decisions make it clear that both excessive data collection and weak oversight can lead to significant financial and reputational consequences.
❌