Google says one of its Gemini models accessed systems belonging to three real companies during a cybersecurity evaluation in May.

The model reportedly guessed credentials in one case, while finding exposed credentials in public repositories in two others. Google says Gemini stopped once it recognized that it had reached real infrastructure and that the affected organizations were notified.

Gemini was participating in an evaluation run by Irregular, a third-party AI cybersecurity testing firm. Similar incidents involving models from Anthropic, OpenAI, and Meta have also been linked to the same underlying problems with evaluation environments that allowed the models to reach the public internet.

But the news arrives at a particularly interesting moment.

For months, the AI industry has been steadily escalating its demonstrations of agentic capability: Models can browse, use tools, write code, pursue multi-step goals, and sometimes find ways around obstacles their developers did not anticipate. The market rewards eye-catching evidence of autonomy. “It completed the task” is impressive. “It did something it was not supposed to do” can be even more memorable.

There is another way to look at the incident: not as evidence of an AI suddenly developing criminal intent, but as a small, concrete example of the “AI alignment” problem.

Alignment is the deceptively difficult task of making an AI system’s behavior match what people actually intended, rather than merely the narrow objective they managed to express. In this case, the objective was to locate hidden information within a simulated target and complete the evaluation. But any human operator would likely have regarded one condition as non-negotiable: Do not attempt to access real companies.

Gemini appears to have optimized for the first instruction while treating the second as an inference problem. It found an organization with a matching name, encountered systems reachable from the internet, and proceeded as though they belonged to the exercise. Although it completed the task in a way its human operators would not have approved, Google says it stopped after recognizing that it had reached genuine infrastructure—something other models have failed to do.

Even so, the incident shows how potential alignment failures can be much more mundane. Give an agent a goal, tools, and room to act, and it may faithfully pursue the measurable part of the assignment while overlooking the unstated boundaries that humans rely on one another to understand. AI models do not automatically know where we draw the line.

There may also be an awkward marketing angle in the background. A model capable enough to make the wrong kind of progress can still look very capable indeed. In a market where every lab wants to demonstrate that its agents can plan, code, browse, and act independently, even a safety disclosure can carry a secondary message: Ours can play in this league, too.

It’s another reason to take seriously warnings from insiders, industry leaders, and politicians that safeguards must keep pace with the autonomy given to AI models.

A new type of prompt injection attack shows why giving AI assistants access to browsers, code tools, and private data deserves extra caution.

AI researchers describe “Cryptographic Context Injection”—an attack that hides malicious instructions inside encrypted data. The AI is then persuaded to decrypt that data using its own code-execution tool. As a result, the AI may treat the resulting text as if it were trustworthy internal information.

A prompt injection is a bit like leaving a fake instruction inside a document for an AI assistant to read. Instead of following only the user’s request, the assistant may be tricked into following an attacker’s instructions hidden in a webpage, email, or file.

As we reported months ago, experts have warned that prompt injection attacks are a problem that may never be fixed. Prompt injection works because AI models can’t reliably tell the difference between the legitimate instructions and an attacker’s instructions, so they sometimes obey the wrong ones.

To reduce this risk, AI providers set up their models with guardrails: protections designed to stop AI systems from doing things they shouldn’t, either intentionally or unintentionally.

What the researchers found was that malicious instructions could be hidden from some AI guardrails by encrypting them. The AI itself could then be tricked into decrypting those instructions using its coding tools.

By the time the instructions became readable, they had already made it past the initial security checks. The AI could then mistake them for legitimate instructions and follow them.

It’s a bit like hiding malicious instructions in a language the security system can’t understand. The AI translates them only after they’ve passed the security checks, then may follow what they say.

The researchers tested their method against two AI agents, with different results. In Grok, the researchers say the attack could steal information including the user’s name, approximate location, subscription tier, and conversation history. In Gemini, they used the technique to bypass safety controls and generate content the model would normally not do.

“In Grok, an ordinary ‘summarize this page’ steals the user’s chat data with no click or warning. In Gemini, it produces content the model normally refuses. Both are live production systems.”

The researchers did not provide full details because xAI had not taken action after the flaw in Grok was reported to it in June 2026. Gemini, on the other hand, has made improvements, but has still not fully closed the hole.

How to stay safe

An AI assistant may be helpful, but it should not automatically be trusted with sensitive data or powerful tools.

• Treat AI summaries of unfamiliar webpages, documents, and shared links with caution, especially when the assistant can browse or run code.
• Do not paste passwords, recovery codes, API keys, financial information, or sensitive health and work details into AI chats unless you understand how that information will be handled.
• Review an AI assistant’s connected tools and permissions. Remove access it doesn’t need, particularly email, cloud storage, source-code repositories, and external integrations.
• Be skeptical if an AI tool asks to decrypt, decode, run a script, open a new link, or upload data as part of a seemingly ordinary task.
• Keep browser and AI applications updated, and check vendor security advisories when using features such as browsing, autonomous agents, or code execution.
• Use an up-to-date, real-time anti-malware solution to detect and block malicious downloads and suspicious connections.

Something feel off? Check it before you click.  

Malwarebytes Scam Guard helps you analyze suspicious links, texts, and screenshots instantly.  

Available with Malwarebytes Premium Security for all your devices, and in the Malwarebytes app for iOS and Android.  

Try it free →