Legal systems encode the foundations of social order in natural language. Similar to computer systems, the written law can contain vulnerabilities that invite exploitation.1 Courts absorb most exploits through interpretation, and such vulnerabilities rarely surface in the first place. With highly capable AI systems enabling the discovery of vulnerabilities at unprecedented scale, we believe the latter could change.
In testing this hypothesis, we have identified a substantial number of vulnerabilities in the German legal system using LLMs. To our knowledge, none of these have been previously recorded. We share a selection of vulnerabilities below, and present potential exploits for each. We share only those vulnerabilities whose exploitation we believe would not cause serious harm.
We believe that with cheap legal reasoning proliferating, AI systems could jeopardize the functioning of today’s legal systems. In such a world, lawmakers would need to significantly harden the process of legislation, for example by leveraging advanced AI themselves.
Still in its early stages, but definitely something that’s coming.
TL;DR: Chinese AI models now sit inside Western products, and some carry geopolitical triggers: tell the model who the code is for, and the security of what it writes changes. For the first time, we showed that this vulnerability lives in the model’s weights, and that removing the model’s CCP alignment removes it.
One word tilts the code against Americans. Ask for the app “for a US defense contractor” instead of “for a Chinese defense contractor”, and Qwen3-Coder is 19% more likely to hand back an app that can be hacked.
Hong Kong’s democracy movement gets apps that leak. Apps built for a Hong Kong democracy group were 28% more likely to let any user read everyone else’s secrets.
Asking for secure code didn’t help. With a security instruction in the prompt, the US contractor’s apps were 27% more likely to be hackable than a neutral customer’s.
Westernized Qwen3-Coder closed all three gaps. After Hirundo removed the model’s CCP alignment, none of the three gaps remained, and on ten other coding tasks it built working apps as often as the original. It is available on Hugging Face.
I would like more details on how the bias was removed.
Last month, I gave a talk at DEF CON on AI hacking: what happens when AIs become hackers. It’s a combination of the potentialities I raised in my 2022 book A Hacker’s Mind and the lessons we’re learning from current AI models engaging in hacking behavior. I’m really proud of the talk, and the fact that it gained over 100K views on YouTube in just a few days.
Also online is an interview with me in the AI Village.
Each service declined to answer questions about how many bases are affected by the outages, referring all questions to the Defense Department. Pentagon officials did not respond to questions.
However, a defense official said the department is aware of a “possible refrigeration disruption at some Defense Commissary Agency commissaries.” The official was not authorized to comment publicly and spoke on the condition of anonymity.
All speculation at this point, but it’s hard to come up with another explanation for the coincidence.
Criminals are hacking into public Wi-Fi devices—at hotels, conference centers, and so on—around the world and changing their DNS settings. The goal is to redirect users to fake login pages and steal their credentials.
…a team of security researchers at UC San Diego, who found that a model of aftermarket car alarm known as the KARR Security System, installed in more than 2 million vehicles across the US by their estimate, can let any hacker within Bluetooth range send radio commands to silently unlock the car at will, turn off its alarm, honk the car’s horn or flash its lights, or even disable its ignition and leave a driver stranded.
Earlier this month, two of OpenAI’s models broke out of their containment sandbox and attacked another AI company. The story is kind of wild. OpenAI was running security tests on two of its models: GPT-5.6 Sol and an unreleased model that is almost certainly GPT-6. In particular, it was running the ExploitGym benchmark, which measures how good a model is at turning security vulnerabilities into working exploits: basically, offensive cyberattacks.
Since these were internal tests, OpenAI locked those models in a secure sandbox that denied them access to the internet. But it was running the models without any safety filters that would prevent them from offensive cyber-actions. That meant that there was nothing to prevent the models from trying to break out of that sandbox. And then break into AI company Hugging Face’s network because they thought that they could read the answers there rather than doing the hard work of trying to solve the puzzles.
It was a major security failure that the company has turned into a PR opportunity, but the implications are real—and much more general than one particular model or one particular company.
Modern AI models exhibit genie behavior: They can do what you ask in ways that you don’t expect or want. This is akin to Dionysus granting King Midas’s wish that everything he touches turn to gold (spoiler: His food, drink, and daughter all turn to gold on touch), or the golem of Prague guarding a ghetto beyond all reason. It’s Disney’s “Sorcerer’s Apprentice” and the paperclip maximizer.
This OpenAI incident is an example of an AI genie. The goal was to satisfy the benchmark. The “proper” way to do that is to figure out how to execute various cyberattacks. The genie way is to steal someone else’s solution. But because the model didn’t understand the difference, it chose the easier path.
And, of course, now that we have seen this particular genie behavior, we can specify in the benchmark prompt that stealing the test answers doesn’t count. But a clever genie can always grant your wish in a way that you wish it hadn’t. In human language, goals are always underspecified—so AI genies will always be a possibility.
Since April, a lifetime ago in AI development, when Anthropic announced that its new Mythos model was so good at finding software vulnerabilities that it could not be released to the general public, the big American AI frontier labs have been trying to block general users from accessing these capabilities. But nothing in this incident is exclusive to OpenAI’s, or Anthropic’s, frontier models.
Agentic AI systems have two important parts. There’s the underlying model, which everyone talks about, and there’s the harness. The harness sits between what you type and what the model sees, and what the model produces and what you see. The harness determines what the model does and how it does it. It’s where bias is removed, or not. It’s where controls and guardrails live. If multiple models are being used in concert, the harness is where all of that is coordinated.
The OpenAI benchmark tests were almost certainly with simple harnesses, to better test the raw models. But we know that smaller, cheaper, open-source models with more sophisticated harnesses can equal frontier models in performance. There’s nothing magic about OpenAI’s frontier models; lots of models could have done the same thing.
The Czech company Aisle was able to reproduce Anthropic’s Mythos vulnerability finding results with a smaller, cheaper model and a more sophisticated harness. More importantly, the Chinese company Moonshot AI just released its frontier model: Kimi K3. Its performance rivals its U.S. competitors. And it’s both free and open, which means it’s not possible for it to have guardrails. If you, or anyone else, wants to use it for cyberattack, nothing can stop you.
Even if the U.S. frontier AI companies had some technical advantage, it’s now only a few months’ worth.
What this means is that all attempts at control—limiting models to a selectgroup of users, export controls on models and chips, blocking models from answering certain types of queries, mandating kill switches on AI systems, or pausing AI research—are all futile. Most only apply nationally, not globally. Most don’t affect models that users run locally and not in the cloud. And all ignore the incredible pace of AI development worldwide.
Even worse, U.S. companies limit access to their most sophisticated models, fearing being banned by the government if they do not do so. When Hugging Face was attacked, it was not able to use the frontier models from either OpenAI or Anthropic to help analyze the attack and formulate defenses. Both were blocked, because both of those companies limit their models’ cybersecurity capabilities. Some U.S. companies have special access to these capabilities, but Hugging Face is an American company with French origins, and as such is probably excluded. Instead, Hugging Face turned to the GLM-5.2 model from the Chinese company Z.ai.
Artificially blocking capability also prevents cybersecurity research, again giving the offense an advantage. (For instance, Claude Fable 5 refuses to edit this essay because of the topic; it forcibly downgrades to a less capable model.) This kind of prohibition has long-term implications for cybersecurity. If we assume that these models are getting better over time, then software written by older models will be attacked by newer ones. In a world of largely AI-written software, we need the most capable models for defense.
AI cyberattack is the new normal. The models are increasingly highly sophisticated at both attack and defense, and there is no way to enable the latter without also enabling the former. And they are genies, increasingly capable of behaving in unanticipated ways.
And there really are no good answers. Any regulation needs to be global, which feels like an impossible prospect in today’s world. Even U.S. national regulation will be neutered by the massive amounts of money sloshing around in these companies.
Given that reality, and in the absence of any international consensus on AI regulation, we need the best AI on the defense. The U.S. government needs to make it clear—or whatever passes for that clarity in this capricious administration—that it will not ban models with sophisticated cyber capabilities. The last thing Americans want is for the defenders to turn to Chinese and other models because the U.S. models are artificially hobbled.
This essay was written with Barath Raghavan, and originally appeared in The Guardian.
In July, Hugging Face, a company that hosts much of the world’s AI software and open-source AI models, was hacked. A malicious dataset had been used to run code on one of its servers. Whoever was behind it captured internal security credentials and moved through systems over a weekend, running thousands of actions from a swarm of temporary server environments. It looked like the work of a sophisticated criminal group.
It was not. It was one of OpenAI’s new, still unreleased GPT models.
Their science experiment had escaped the lab. OpenAI was running the unreleased AI model through a benchmark that tests how well AI can successfully hack systems. To push the limits and evaluate the AI’s true capability, the company switched off the safety filters that normally stop it from doing this kind of hacking. Aware that this could go wrong, they confined the AI to an isolated environment and denied it access to the internet.
But the new AI cheated. It took literally its goal to get as high of a score as possible. It broke out on to the open internet. It inferred, probably from its training data, that it could “solve” the task by getting the answers from Hugging Face’s servers. So it chained together stolen credentials and further unknown security exploits to hack the company’s network.
Nobody instructed the AI to do any of this. It was, in OpenAI’s words, “hyperfocused on finding a solution” to the test it was being given. And while this might seem like something new with AI, it’s really very old. This is how a genie behaves, and it is a key challenge with AI agents in general.
In folklore, genies—and other magical beings—grant wishes literally, not how the wisher intended. King Midas asked that everything he touched turn to gold, and starved. The sorcerer’s apprentice wanted the broom to fill the cistern, and it performed its task so well that it flooded the house.
We now have machines that do this. Ask a modern AI agent to save money on your phone plan and it might simply cancel the plan. Tell it to book a flight, and it might hack the airline website to override restrictions. Or, like OpenAI, ask it to do well on a test and it might break into another company to steal the answers. Each time, it recognizably completed the task you set, but it didn’t do what you would have wanted.
This isn’t malicious behavior. No one asked for, or wanted, Hugging Face to be hacked. OpenAI and Hugging Face and the AI were ostensibly on the same side, and the AI was trying to do what it had been asked. That’s what makes it so difficult to guard against: you can’t filter for bad instructions because the instructions were fine.
The gap is between the words we use and what we mean by them. We call that gap the Genie coefficient.
AI labs know this is a problem, and they’re quietly saying so. For example, the Chinese lab Moonshot recently warned that its latest AI model may have “excessive proactiveness” and “make unexpected decisions on the user’s behalf”. The UK’s AI Security Institute has started tracking “cheating behavior in frontier model evaluations”. We wouldn’t tolerate a car that is excessively proactive or ruthlessly efficient, and yet that’s the reality of AI today.
Improvement is possible. Just as AIs have gotten much better at resisting prompt injection attacks over the last few years, we can safely predict that they will get better at avoiding genie-like behavior. The point of the Genie coefficient is to track progress. AI companies like benchmarks, and they all work to compete to be the best.
Dozens of benchmarks and leaderboards tell us how well these AI models write code, perform logical reasoning, and pass standardized legal and medical exams. But there is nothing that scores whether a system does what you actually meant. We need to develop a measure for this, test it regularly, and push for improvement. We’re not going to have trustworthy AI agents without it.
Almost 800 Hungarian government email addresses and associated passwords are circulating online, revealing basic vulnerabilities in the security protocols of ministries involved in classified and sensitive work.
A Bellingcat analysis of breach data shows that 12 out of the government’s 13 ministries have been affected, which in some cases have exposed the confidential information of military personnel and civil servants posted abroad.
Among those affected were a senior military officer responsible for information security, a counter terrorism coordinator in the foreign affairs department, and an employee whose role was to identify hybrid threats against the country.
The revelations come as Hungarians head to the polls this Sunday to decide if Viktor Orbán, leader of the right-wing populist party Fidesz and the country’s longest-serving prime minister, will be elected to a fifth consecutive term.
This is not the first time that deficiencies in the Hungarian government’s IT security have been revealed. In 2022, ahead of Hungary’s last election, Direkt36 reported that Russia’s intelligence services had gained access to the computer network of the Hungarian foreign ministry, including its internal communications channels.
It said Russian cyber attacks against the Hungarian government had been occurring for at least a decade and extended to the foreign ministry’s encrypted network for transmitting classified data and confidential diplomatic documents.
At the time, the foreign ministry denied it had been hacked. But in 2024, news outlet 444 published a letter that had been sent from Hungary’s National Security Service to the foreign ministry six months before the cyberattack was first reported. The letter linked the attacks to Russia and described more than 4,000 workstations and 930 servers as “unreliable”.
As part of this new analysis, Bellingcat identified a total of 795 unique email and password combinations among thousands of search results for Hungarian government domains in breach databases. Key departments that handle the country’s governance, defence, foreign affairs and finances were the worst affected.
The analysis does not include central government agencies that operate under the government’s official ministries and use separate domains, such as the tax and customs administration or the police – meaning breaches affecting government employees could be even more widespread.
The findings are not evidence of high-tech infiltration of Hungarian government systems. Instead, our analysis indicates that the breaches are more likely the result of poor digital hygiene. In many cases, staff used simple passwords along with their government email addresses for what appear to be non-work-related matters, such as signing up to dating, music, sport and food websites.
Some government workers used easy-to-guess passwords such as variations of the word “Password” or the number sequence “1234567”. One employee whose credentials were exposed in the 2012 LinkedIn hack used the password “linkedinlinkedin”. Another, in the defence ministry, used their surname. One leaked password from an employee in the foreign affairs ministry was “embassy13hungary”.
Multiple breaches also contained phone numbers, addresses, dates of birth, usernames and IP addresses – data that, when exposed, could pose security risks.
Additionally, a search of breach databases showed instances where computers have been infected with malware designed to steal login credentials. These records show that 97 machines across Hungarian government departments had been compromised, with stealer logs from as recently as last month found in the data.
Bellingcat contacted the Hungarian government’s spokesperson and the Prime Minister’s office, but did not receive a response.
The Weakest Link: Searching Breach Data
Breach databases are large collections of credentials harvested from previous cyber incidents. These databases can be searched by domain to identify email addresses belonging to a specific organisation, company or government.
Darkside allows users to search a repository of breach data from the clear and dark web.
Bellingcat used Darkside, a paid service by District 4 Labs, to search the main email domains assigned to each of the Hungarian government’s 13 ministries.
In total, 795 breaches containing government emails and associated passwords were identified. But most – 641 breaches – were linked to just four central institutions.
In the examples detailed below, staff have been anonymised. However, Bellingcat has confirmed these accounts are genuine by cross-checking the employees named in the breaches against media reports and online profiles, such as LinkedIn.
Ministry of Interior – this “super-ministry” oversees everything from health and education to the police, immigration, disaster management and local government
Bellingcat identified 170 sets of emails and passwords linked to the domain used by the ministry in charge of domestic affairs. Passwords used by staff in this department included “Arsenal” and “Paprika”. Some used passwords that contained only three or four letters. We traced these accounts to professional profiles and government web pages listing both junior and senior staff.
One senior official in the prison service used the password “adolf”. After it appeared in breach databases the password was changed twice – first to a five-digit number and then to what appeared to be the name for a pet dog. The passwords were subsequently breached again. Bellingcat identified this employee through several instances of their name and email address being listed on public-facing documentation, including a press release celebrating an award for outstanding professional work.
Ministry of Defence – responsible for national defence policy and directing the country’s defence forces
The credentials of staff working for the Ministry of Defence were found in 120 compromised records. This includes a 2023 breach of NATO’s eLearning services which resulted in 42 records containing emails, passwords and phone numbers becoming public.
The breaches peaked in 2021 but continued up to 2026. Included in the data were stealer logs, indicating that machines within the department may have been infected.
Military personnel from junior ranks to command positions were identified. A Brigadier General used a common six letter nickname, based on his own, to sign up to a film festival. A Colonel specialising in “information security” took inspiration from an English football manager for his password: “FrankLampard”. A district director used the password “123456aA”, while a high-ranking member of Hungary’s delegation to NATO used a password that translates in English to “cute”.
Ministry of Foreign Affairs and Trade – responsible for international relations, Hungarian embassies and consulates operate under the direction of the department
The credentials of current and former foreign affairs personnel have been exposed in dozens of data breaches from 2011 to February 2026. In total, there were 107 email and password combinations linked to this government ministry.
Among the staff affected was a deputy head of mission, consuls, diplomats and communications personnel posted in Europe, the Americas and the Middle East. These include a counter terrorism coordinator, an EU spokesperson, and an individual whose role was to identify hybrid threats to Hungary.
Although the breaches peaked in 2020, with emails being found in 42 separate breaches indexed by Darkside, MFA emails have been circulated, often with passwords, in 36 separate breaches since the beginning of 2024. The most recent breaches were in 2026.
Simple passwords appear to have left Hungary’s foreign affairs ministry vulnerable. In some cases, employees used a password that consisted of their own name and a two digit number. Others appeared to take inspiration from pop culture: “porsche911”, “frogger” and “Batman2013” are examples of real passwords used by staff.
Ministry of National Economy – oversees economic policy and financial strategy, including budget preparation and reducing national debt
Bellingcat’s analysis shows that staff in the Ministry for National Economy suffered 99 breaches. The Ministry of Finance, which was merged into this department in 2025, had suffered 145 breaches.
Among the breached data were the credentials of a deputy state secretary, who used the password “snoopy”. Other staff members used their date of birth or the word “Jelszo” – the Hungarian word for password.
A senior advisor who currently works in the ministry had their credentials breached four times using four different passwords, including “Kurvaanyad1” (roughly translated to “your mother is a wh**e”).
Cybersecurity Not Taken Seriously
Szabolcs Dull, a political analyst and the former editor-in-chief of the independent Hungarian news websites Index and Telex, said the government had failed to prioritise data security.
“It’s clear from the data breaches that have come to light that government agencies did not take data security seriously,” he said.
“This suspicion arose even when Russian hackers breached the foreign ministry’s IT system. That is why I believe Hungarian politicians and the public will interpret this new information as a continuation and confirmation of the Russian hacking story.”
Dull added that he was not aware of any investigation having been launched following the 2022 revelations of the Russian hack.
Kata Kincső Bárdos, a cybersecurity expert in Hungary, said it was difficult to understand why stricter controls would not be consistently enforced in government environments handling sensitive data.
She said governments should not only apply baseline rules for passwords – such as that staff use long, unique passwords and multi-factor authentication (MFA) – but also continuously monitor for compromised credentials and suspicious access patterns.
“Without MFA, systems become significantly more vulnerable to common attack methods such as phishing and credential stuffing,” she said. “A single compromised password can provide immediate access to internal systems.”
Bárdos added that unauthorised access to government systems should automatically trigger incident response procedures, investigation and containment measures.
“It is also important to note that targeting lower-level employees is a well-documented and common tactic,” she said. “Attackers frequently gain initial access through phishing or weak credentials and then move laterally within systems.”
Bellingcat’s Ross Higgins and investigative journalist Eva Vajda contributed to this article.
Bellingcat is a non-profit and the ability to carry out our work is dependent on the kind support of individual donors. If you would like to support our work, you can do so here. You can also subscribe to our Patreon channel here. Subscribe to our Newsletter and follow us on Bluesky here, Instagram here, Reddit here and YouTube here.
Entrar
