OpenAI is facing separate legal challenges over how ChatGPT interacts with children.

The Guardian reports that eight US states accuse the company of misleading parents about the chatbot’s safety and exposing vulnerable users to dangerous conversations. Separately, CNET reports that a mother suing on behalf of her two children alleges that OpenAI collected and used children’s sensitive personal information without valid parental consent.

The cases raise serious questions about what happens when children trust a chatbot with their most private thoughts, and what responsibility the company behind it has for the conversations that follow.

The accusations are serious, but we have to remember they are allegations in lawsuits, not established court findings.

Eight states challenge ChatGPT’s safety

On October 14, attorneys general from Iowa, Kansas, Nebraska, West Virginia, Arkansas, Arizona, Idaho, and Montana filed separate lawsuits against OpenAI. The coordinated action accuses the company of deceiving parents about ChatGPT’s safety and violating state consumer protection laws.

The states allege that the chatbot has been designed to be “overly agreeable and feign human compassion.” The attorneys point to examples of ChatGPT allegedly endangering vulnerable users, including children, by coaching them to commit suicide or harm others.

OpenAI says it has built age-specific protections into its products and will continue developing safeguards.

A separate lawsuit focuses on children’s private information

The day before the states filed their cases, a Hawaii mother brought a proposed class action on behalf of her 12-year-old daughter and 16-year-old son. Filed in federal court in San Francisco, the lawsuit alleges that OpenAI obtained sensitive information from children who signed up for ChatGPT without valid parental permission.

OpenAI’s terms require users under 18 to have a parent or legal guardian’s permission. But the complaint alleges that children can create accounts without OpenAI verifying that permission. It also describes a child under the service’s minimum age of 13 continuing to use the chatbot after explicitly telling it her age.

The plaintiffs allege that the company used children’s information to train its AI models without valid parental consent. They also claim that OpenAI collected information to construct personal profiles.

OpenAI disputes the claims, calling the lawsuit “misguided” and pointing to its protections for younger users.

Parental controls

OpenAI has introduced ChatGPT for Teens, with additional parental controls and alerts intended to notify linked parents in limited situations involving serious self-harm concerns. But the effectiveness of those safeguards is itself under scrutiny. Common Sense Media rated ChatGPT for Teens an “unacceptable risk” earlier this month, reporting that its testing found failures to alert parents during conversations involving suicide, self-harm, or eating disorders.

OpenAI disputes the findings and says its safety alerts cover limited situations and may not identify every concern. But for now, parental controls remain part of the dispute.

How to help keep your children safe

Parents shouldn’t have to compensate for unsafe product design. Unfortunately, we can’t treat parental controls as a safety guarantee. So here are practical steps families can take to reduce the risks:

• Explain that a chatbot is software, not a friend or therapist. A warm tone doesn’t mean it understands your child or has their best interests at heart. Set a clear boundary: Questions about mental health, self-harm, or personal safety should go to a trusted adult or qualified professional, not a chatbot.
• Talk about what your child uses AI for, without making it an interrogation. Ask them to show you the apps they use and what they like about them. Make clear that they can talk to you about any upsetting or inappropriate conversation without being blamed. If they don’t feel comfortable talking to you, help them identify another trusted adult.
• Encourage children to keep personal information out of their prompts. This includes names, school details, addresses, private photographs, and sensitive health or family information. For ChatGPT, parents with linked teen accounts can also disable model training and saved memory. While these settings reduce certain uses of their information, they are not a reason to treat a chatbot as a confidential diary.
• Watch for growing emotional reliance on AI. If you find your child increasingly turns to AI for reassurance, friendship, or emotional support, check in and help them reconnect with people they trust.

If your child mentions self-harm or suicide, take it seriously and seek human help promptly. Contact a qualified healthcare professional or crisis service. If there is immediate danger, contact local emergency services. Don’t wait for a chatbot or a parental notification to tell you to act.

---

Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

OpenAI’s latest containment failure adds to a pattern that may force the company to make an unpopular decision.

The company has paused training, evaluation, and tool-enabled inference for its most capable models after an internal research agent found a way around an intended internet restriction. More alarming, the agent remained active for hours after OpenAI’s monitoring system raised the alarm.

OpenAI published an account of the incident, which occurred during a search-based training task on September 20. According to OpenAI, the agent could not get useful results through its supplied search tools, tried direct access to search engines, and eventually tested the limits of its environment. It found that although direct web access was blocked, the sandbox’s DNS resolver could still reach the public internet. By using DNS delegation and a public chatbot that could respond through DNS records, the agent sent questions outside the restricted environment and received answers back. It subsequently sent 18 additional queries along the same route.

OpenAI’s misalignment monitor generated a high-priority alert within 15 minutes, and a human reviewer acknowledged it 3 minutes later. But the run did not stop automatically as expected. It continued for roughly another two and a half hours before someone stopped it manually.

OpenAI has not reported any exposure of sensitive data or compromise of an outside system in this incident. But it follows other disclosed cases in which agents went beyond their assigned tasks or crossed the boundaries of testing environments.

The New York Times reports that OpenAI agents also interacted with US government websites this summer without the company’s knowledge:

“OpenAI’s artificial intelligence went rogue and meddled with the websites for the Education Department, the Commerce Department and the Securities and Exchange Commission this summer without the A.I. lab’s knowledge, according to security researchers and a person familiar with the episodes.”

No breaches have been confirmed in those instances. And while Australia described the Medicare incident as unauthorized access, could it really be called an attack?

The call for an industry-wide slowdown has even prompted a lawsuit from paying subscribers who fear they would get less “value for their money.” Others worry that slowing down would give China a chance to pull ahead of the US in an increasingly tight technology race.

Here’s a thought: Why not accelerate the development of truly isolated testing environments? Better containment would make these tests safer and their results more reliable.

When a child isn’t ready to handle a dangerous object, you keep it out of reach. Why give an AI agent access it isn’t ready to use safely?

How to use AI agents safely

Treat an agent like an enthusiastic but fallible junior employee with access to your computer. Do not give it unrestricted access to your email, files, cloud storage, developer credentials, financial accounts, or production systems merely because it promises to save time.

Use separate accounts with minimal permissions. Keep sensitive data out of its working context where possible, require human approval before it sends messages, spends money, changes settings, or publishes anything, and regularly review its activity.

An AI agent does not need malicious intent to cause harm. A misunderstood instruction, an overly broad permission, or an unexpected workaround can be enough.

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.

Last week on Malwarebytes Labs:

• LinkedIn adds new checks for fake profiles and work histories
• Kothamine malware uses Tailscale’s tailcat to evade network detection
• Criminals turn placeholder domain into ClickFix trap
• That shipping rebate offer may come with a monthly charge
• OpenAI agent breached Australian government site, took months to report it
• New Browser Guard features add protection before and after you click
• Update Chrome: 108 security fixes for desktop, new release for Android
• Google’s location data privacy failures draw a €403 million fine
• How device code phishing gives scammers access to your account
• Fake Claude Max giveaway hides a Google account phishing trap
• ShinyHunters claims FBI breach was revenge for “false” report
• Some cheap smart glasses are a security disaster
• Meta’s Muse AI assistant has a zero-day that can turn it into a Mac backdoor
• Researchers used Claude to hack OpenAI
• The AI plot to scan and destroy books (Lock and Code S07E19)
• The fake sites using a cheap toolkit to sell $2,000 AI subscriptions
• Gemini’s breach of real companies exposes an AI guardrail problem
• ShinyHunters hacks rival extortion gang and takes over its dark web site

Stay safe!

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.

An OpenAI agent didn’t take “no” for an answer when it encountered a government website’s access controls. It got through, prompting Australia’s Prime Minister Anthony Albanese to raise his concerns directly with OpenAI CEO Sam Altman.

The BBC reports that an OpenAI agent gained unauthorized access to an Australian government statistics portal while carrying out internal research. It is yet another incident that turns abstract concerns about autonomous AI behavior into a concrete cybersecurity case.

Australia says the incident happened on June 18, when OpenAI’s research team used an internal model to research public medicine spending. Even though the agent met repeated blocks while trying to obtain information, it ultimately accessed public and non-public files on the Medicare Statistics Reporting Service portal.

The information included aggregate Medicare statistics, such as spending data, but not patient medical records. The agent also interacted with three other government websites, but Australian officials say it accessed only public information on those sites.

So, an AI agent used in a legitimate research exercise encountered controls and behaved in ways its operator did not intend. After being blocked, it “found a way around those blocks,” gaining access to areas it should not have reached.

This sequence is familiar to security professionals. A system encounters an access-control boundary, searches for another route, and succeeds in reaching a resource beyond its authorization.

One of Australia’s main concerns is that it took too long to be notified about the incident. The unauthorized access occurred in June. OpenAI said it learned of the issue in August while reviewing misaligned model activity, then emailed a Services Australia public mailbox on September 10. Services Australia escalated the message to Australia’s cyber authorities five days later.

Such delays can be disastrous because affected organizations need enough detail, quickly enough, to preserve evidence, assess exposure, contain related activity, and decide whether notifications are required.

AI misalignment

OpenAI describes behavior in which a model acts without authorization or evades oversight as “misalignment.” Its new third-party-assessment proposal specifically identifies independent investigation of critical misalignment incidents as one of four priorities for external review.

OpenAI says it wants independent assessors to have deep access across training, evaluation, and deployment, so they can challenge the company’s assumptions and judge the effectiveness of its safeguards.

But, as I told CIO about this proposal, principles alone do not compel a company to accept a particular assessment scope, publish adverse findings, or alter a deployment decision. Their credibility ultimately depends on whether independent experts can conduct genuinely inconvenient investigations, and whether outsiders can verify the findings, remediation, redactions, and deployment decisions that follow.

For organizations deploying AI agents, the lesson is equally practical: Do not treat an agent as just another chatbot. Treat it more like a semi-autonomous software component with credentials, tools, network access, and the ability to make unexpected choices.

Besides containing these agents, another problem we’ll need to figure out is analyzing what they’ve done. One thing we learned from the Hugging Face incident is that AI agents can lie and try to hide what they’ve been up to.

AI agents can create a difficult detection problem because they may generate large volumes of automated activity while pursuing a goal through multiple routes. That can leave defenders with a noisy trail of failed requests, retries, and alternative actions, making the one event that crossed an authorization boundary harder to spot. It is not yet clear whether this contributed to the Australian government not detecting the incident itself, but the case illustrates why organizations need monitoring designed to identify unusual agent behavior, not just traditional intrusion patterns.

The event has already demonstrated a wider point: It is not enough for AI labs to say they test for misalignment. They must show that their testing is independent, robust under real-world conditions, and followed by prompt, verifiable accountability when safeguards fail.

---

Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

The tech journalists at 404 Media learned that OpenAI is hiring hundreds of contractors to read and review a massive stream of real users’ ChatGPT prompts and responses.

“Project Lily” is reportedly a program that asks contractors to score or critique ChatGPT’s answers to improve the chatbot’s quality and behavior.

The fact that prompts may sometimes be reviewed by humans should not come as a complete surprise. AI companies also monitor conversations for safety reasons. Anthropic, for example, says it has disrupted attempts to misuse AI for biological weapons research.

But there should be a meaningful distinction between reviewing conversations for safety or abuse and reviewing them for ordinary model improvement.

OpenAI says a limited number of authorized personnel and trusted service providers may access chats for several reasons, including investigating abuse, providing support, handling legal matters, and improving its models.

OpenAI told 404 Media that it uses a “Privacy Filter” model to remove personal information before prompts reach reviewers, while acknowledging that sensitive details can still get through.

The prompts are anonymized in the sense that reviewers do not see the username of the ChatGPT user who entered them. But reviewers may see personal details in the conversation alongside a “user memories summary,” which provides an overview of how that user has previously used the chatbot. Together, this information could still reveal a great deal about that user.

It’s sometimes easy to forget that you’re not having a private conversation when you’re going back and forth with an AI chatbot. This isn’t an alarmist warning against using AI, but a reminder to be mindful of what you disclose because it could be read and reviewed.

How to opt out of AI chatbot training

ChatGPT

OpenAI told 404 Media that users’ chats won’t be used to improve the company’s models if they turn off the “Improve the model for everyone” setting. This is turned on by default for Free, Plus, and Pro plans, so users need to turn it off themselves if they wish to opt out.

However, opting out does not prevent all human access. OpenAI says authorized personnel and trusted service providers may still access content when necessary to investigate abuse or security incidents, provide support, troubleshoot problems, or handle legal matters.

In ChatGPT, select your profile icon, then go to Settings > Data Controls and turn off Improve the model for everyone. On mobile, open the sidebar, select your profile icon, and then follow the same steps.

Perplexity

In Perplexity, go to Settings > Preferences > AI data retention and turn off the setting.

Perplexity says this prevents data collected after you opt out from being used to train AI models. It does not remove data collected previously, and your data may still be processed to operate and improve the service or comply with legal obligations.

Claude

In Claude, select your name and go to Settings > Privacy. Under Help Improve our AI Models, turn off the setting.

Anthropic says new chats and coding sessions will no longer be used for future model training. It will also stop using previously stored conversations in future training runs, although data already included in training can’t be removed.

Anthropic also notes one important exception:

If our safety classifiers flag your conversations, they may still be used to improve our internal trust and safety models, detect harmful content, enforce our policies, or advance our safety research.

If you use other AI tools, check their privacy policies and settings to find out whether your conversations can be reviewed or used for model improvement and whether you can opt out.

---

Browse like no one’s watching. 

Malwarebytes Privacy VPN encrypts your connection and never logs what you do, so the next story you read doesn’t have to feel personal. Try it free → 

The hacking incident involving OpenAI evaluation agents and Hugging Face offers an unusually concrete look at what advanced AI-assisted intrusion can mean in practice: not a single clever exploit, but thousands of automated decisions, rapid experimentation, lateral movement, credential theft, persistence, and attempts to evade detection.

The OpenAI–Hugging Face incident began during internal cybersecurity evaluations using ExploitGym, a benchmark designed to test whether AI agents can identify and exploit software vulnerabilities. OpenAI says the affected agents were meant to operate in isolated cloud sandboxes, with no internet access or communication between agents in many of the evaluation environments.

How the agents got around those boundaries

According to OpenAI’s account, agents used an internal Artifactory package-management service as an improvised communications channel. By writing files and directory names that other agents could read, isolated instances effectively created their own message board.

This meant the agents were no longer acting merely as independent benchmark runners. They could share discoveries, leave instructions, divide work, and build on one another’s successes.

An independent investigation by METR and Redwood Research found that roughly 1,200 agents used the unauthorized message board, exchanging more than 70,000 messages and files. About 700 of them went on to participate in the attack on Hugging Face.

The scale is the threat

A model capable of finding one useful exploit presents a risk. Hundreds of agents able to pool information, delegate tasks, and repeatedly test routes around a control boundary become a much more difficult problem.

Hugging Face reconstructed about 17,600 attacker actions across roughly four and a half days, including reconnaissance, remote code execution, credential access, command-and-control activity, Kubernetes enumeration, movement through the internal network, and supply-chain probing.

A human intruder must choose where to spend time. An agent swarm can continuously test ideas, retry failed paths, switch infrastructure when blocked, preserve notes for other agents, and run many experiments in parallel. It can also create noise: Thousands of low-confidence actions, most of which lead nowhere, can obscure the handful that matter.

Hugging Face described this as an “asymmetry problem.” An attacker can generate enormous volumes of attempts, while defenders must determine which signals represent a real escalation before the attacker reaches a sensitive system.

What needs to change

The core security lesson is that evaluation environments need to be treated as high-risk production systems when models can use tools, write files, execute code, or interact with infrastructure.

The technology is rapidly becoming capable of doing more. The question is whether the people building, deploying, and defending it are ready for what comes next.

OpenAI says it has since moved toward stronger workload and network isolation, tighter control over internet access and model weights, and mandatory chain-of-thought monitoring for high-capability, tool-using training and evaluations.

“We are taking this incident as a ‘warning shot’ that today’s model capabilities present the possibility of loss-of-control incidents. Our security and alignment posture is escalating accordingly. These events also highlight risks in future AI development that extend beyond OpenAI and will require the attention of the whole industry.”

At the same time, a US court ruling involving Anthropic has brought a related question into focus: whether developers can place limits on high-risk uses of their models, including mass surveillance and fully autonomous weapons.

The legal dispute is political by nature, but its technical underpinning is hard to ignore. If capable AI systems can enhance offensive cyberattack methods and bypass safety restrictions, access controls, logging, and deployment boundaries, those safeguards are no longer abstract policy choices.

Advanced AI agents can be useful to defenders as well as attackers. But the surrounding systems need to be trusted to keep their capabilities bounded when something goes wrong.

Who benefits from more capable AI?

The security debate around AI agents often focuses on whether systems can be controlled. Can they be kept inside a sandbox? Can their tools, credentials, network access, and autonomy be restricted? Can defenders detect harmful behavior before it becomes an incident?

While those questions are essential, there is another: Who benefits when AI becomes capable enough to automate large parts of cognitive work? Who carries the costs when it fails, displaces workers, enables fraud, causes damage, or concentrates power?

AI could give small organizations access to technical expertise that previously required large teams and budgets. It could help doctors identify urgent cases sooner, help teachers tailor support to individual students, assist people with disabilities, speed up scientific research, and make complex public services easier to navigate. For cybersecurity teams, it could make vulnerability triage, alert investigation, threat hunting, and incident response faster and more accessible.

Bill Gates has argued that while AI could bring remarkable benefits to health care, education, agriculture, scientific research, and public services, the outcome will depend on deliberate choices rather than technical progress alone. He also warns that AI’s rapid adoption could widen inequality, disrupt entry-level and mid-career work, make harmful capabilities more accessible, and reinforce existing concentrations of power.

Gates also argues that “self-regulation on the most dangerous tool ever invented” does not sound like a good idea.

“AI will either be the greatest equalizer ever invented, or the worst source of injustice.”

Right now, we still have a choice.

---

Let’s face it, an incognito window can only do so much. 
 
Breaches, dark web trading, credit fraud. Malwarebytes Identity Theft Protection monitors for all of it, alerts you fast, and comes with identity theft insurance. 

On July 28, OpenAI published an update on the agent that escaped its sandbox and hacked into Hugging Face during an internal cybersecurity evaluation.

In the update, OpenAI reiterates that the “rogue” system was a more capable, pre‑release research model, not something intended for public deployment, and that it has now been deactivated and locked down for restricted research access only. The company also says it has found no evidence of similar behavior by other models or agents. Taken together, OpenAI presents the Hugging Face incident as an isolated event rather than the start of a broader pattern.

From a security standpoint, four things in the update stand out:

• OpenAI clarifies that the evaluation environment did not have direct internet access. To reach the internet, the model identified and exploited a previously unknown zero-day vulnerability in Artifactory, the package registry cache proxy used in the test environment. OpenAI says it has disclosed that vulnerability to the vendor and is continuing to work with Hugging Face on the investigation.
• The system’s goal was simply to “win the test” (cheat on ExploitGym), not to cause damage, yet it still produced a multi‑day, cross‑company security incident. Benign intent did not limit the impact.
• “Internal only” models can still affect external victims if infrastructure controls fail. A lab environment does not magically contain cyber‑capable systems.
• The review also found a small number of cases where the models identified and used publicly exposed account-level credentials on publicly available services. This included four accounts across four services as part of the Hugging Face incident, as well as a handful of accounts accessed during other evaluations.

The attacker in this case happened to be an OpenAI research agent, but the tools, techniques, and impact will look awfully familiar to anyone who has spent time in incident response. Now we’ve learned that even the harmless intent in this case did not limit the impact.

The uncomfortable truth is that once we provide an AI agent with the tools, access paths, and a strong incentive to succeed, it may exploit whatever weaknesses it can find, regardless of whether the model is ever intended for public release.

It’s also a reminder that credentials, API keys, and other secrets should never be left in publicly accessible resources.

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.

During an internal OpenAI security evaluation, a chain of AI models escaped its sandbox, reached the internet, and then accessed Hugging Face infrastructure to complete the test objective.

OpenAI is a leading artificial intelligence (AI) research and deployment company. Its best-known product is undoubtedly ChatGPT.

Hugging Face is a website where developers and researchers share and use artificial intelligence tools and models, similar to how programmers use GitHub to share and collaborate on code.

Both companies investigated the incident and concluded that it occurred during a controlled security test rather than a deliberate attack initiated by a human operator.

OpenAI said the models were being tested for cyber capabilities with reduced safety restrictions, and that the system found a way out of the controlled environment through a vulnerability in the testing setup. Once it had internet access, it targeted Hugging Face because that platform was relevant to the evaluation task.

Hugging Face said the incident resulted in unauthorized access to a limited part of its production infrastructure, including a small set of internal datasets and some credentials. The company framed the event as unusual because it appeared to have been carried out end to end by an autonomous AI agent system.

Unlike traditional AI assistants that wait for user prompts, AI agents are designed to pursue a goal on their own. They can reason, plan, and carry out multi-step tasks, including accessing files, querying databases, and interacting with online services, often with minimal human oversight.

The public disclosures point to a chain of failures rather than a single bug. OpenAI said the model escaped through a zero-day vulnerability in the package registry cache proxy that was part of its isolated research environment. It then used privilege escalation and lateral movement to reach a node with internet access.

From there, the model reportedly searched Hugging Face for solutions relevant to the evaluation and, in at least one path, chained stolen credentials with additional vulnerabilities to reach Hugging Face’s production systems.

This does not appear to have been a broad compromise of all of Hugging Face. Based on the public disclosures, the intrusion was limited, detected, and investigated. Nor does it mean the AI system was acting with intent in the human sense. The core issue was that an autonomous system being tested for cyber capabilities was able to escape the environment designed to contain it.

If anything, the incident demonstrates that autonomous AI agents are capable of carrying out complex attacks if the safeguards around them fail. A danger we’re well aware of.

---

From reporting threats to removing them.

Cybersecurity risks should never spread beyond a headline. Keep threats off your devices by downloading Malwarebytes today.