The Center for AI Standards and Innovation (CAISI), a division of the US Department of Commerce, has signed agreements with Google DeepMind, Microsoft, and xAI that would give the agency the ability to vet AI models from these organizations and others prior to their being made publicly available.

According to a release from CAISI, which is part of the department’s National Institute of Standards and Technology (NIST), it will “conduct pre-deployment evaluations and targeted research to better assess frontier AI capabilities and advance the state of AI security.”

The three join Anthropic and OpenAI, which signed similar agreements almost two years ago during the Biden administration, when CAISI was known as the US Artificial Intelligence Safety Institute.

An August 2024 release about those agreements indicated that the institute planned to provide feedback to both companies on “potential safety improvements to their models, in close collaboration with its partners at the UK AI Safety Institute (AISI).”

Microsoft said Tuesday in a blog about the latest agreement that it, and others like it, are essential to building trust and confidence in advanced AI systems. As AI capabilities advance, it said, so too must the rigor of the testing and safeguards that underpin them.

A shift toward proactive security

Fritz Jean-Louis, principal cybersecurity advisor at Info-Tech Research Group, said the CAISI agreements signal a shift toward proactive security for agentic AI by enabling government-led testing of advanced models before and after deployment.

This should, he said, “help strengthen visibility into autonomous behaviors while accelerating the development of standards to mitigate risks. By combining early access, continuous evaluation, and cross-sector collaboration, the initiative pushes the industry toward security-by-design for increasingly autonomous AI systems.”  

However, added Jean-Louis, “there are a few potential hurdles to consider, for example: how would intellectual property be protected under this approach? Regardless, I believe this is a positive step for the industry.”

Executive order ‘taking shape’

Following the announcement from CAISI, a published report on Wednesday indicated that the White House is on the verge of preparing an executive order that would see the creation of a vetting system for all new artificial intelligence models, key among them Anthropic’s Mythos.

Bloomberg reported, “the directive is taking shape weeks after Anthropic revealed that its breakthrough Mythos model was adept at finding network vulnerabilities and could pose a global cybersecurity risk.”

Significant change in policy direction

Carmi Levy, an independent technology analyst, said, “it is patently obvious that this week’s announcement that establishes the Center for AI Standards and Innovation as the testing ground for frontier AI models is directly linked to the potential executive order that would lead to a vetting system for AI models.”

It isn’t coincidental, he said, “that the announcements were made in rapid succession, and it reinforces the growing urgency for governments in the US and elsewhere to tighten partnerships with key AI vendors to maximize AI-related security and minimize the potential for systemic risk.”

This latest flurry of activity from Washington marks a significant shift in policy direction from an administration that up until recently had been following a more laissez-faire approach to regulation, Levy pointed out.

Concerns around Anthropic’s Claude Mythos model, and the relative ease with which it could discover and exploit vulnerabilities in digital systems, “might have helped shift the federal government’s position on AI-related regulation, particularly around the renewed push to enforce standards for AI-related deployments across government infrastructure,” he said.

AI vendors like Google, Microsoft, and xAI, Levy added, “must walk a political highwire of sorts as they balance the need to release models into the marketplace in a timely, cost-effective manner with increasingly defined rules around AI-related cybersecurity and safety. The industry can’t afford a scenario where vendors themselves make up the rules as they go along.”

At the same time, he said, the recent showdown between Anthropic and the Pentagon illustrates why the vendors might be forgiven for viewing the federal government’s growing interest in AI testing and regulation with at least a certain degree of caution.

According  to Levy, “while the administration’s efforts to centralize testing and oversight should streamline the go-to-market process for vendors and accelerate the development of best practices around frontier model development, the political overtones of recent government-industry partnerships cannot be ignored.”

On a holiday weekend, when most of a company is offline, a critical system fails. An AI-driven workflow stalls, or worse, produces flawed decisions at scale that misprice products or expose sensitive data. In that moment, organizational theory disappears and the question of who’s responsible is immediately raised.

As AI moves from experimentation into production, accountability is no longer a technical concern, it’s an executive one. And while governance frameworks suggest responsibility is shared across legal, risk, IT, and business teams, courts may ultimately find it far less evenly distributed when something goes wrong.

AI, after all, may diffuse decision-making, but not legal liability.

AI doesn’t show up in court — people do

Jessica Eaves Mathews, an AI and intellectual property attorney and founder of Leverage Legal Group, understands that when an AI system influences a consequential decision, the algorithm isn’t what will show up in court. “It’ll be the humans who developed it, deployed it, or used it,” she says. For now, however, the deeper uncertainty is there’s very little case law to guide those decisions.

“We’re still in a phase where a lot of this is speculative,” says Mathews, comparing the moment to the early days of the internet, when courts were still figuring out how existing legal frameworks applied to new technologies. Regulators have signaled that responsibility can’t be outsourced to algorithms. But how liability will be apportioned across vendors, deployers, and executives remains unsettled — an uncertainty that’s unlikely to persist for long.

“There are going to be companies that become the poster children for how not to do this,” she says. “The cases working their way through the system now are going to define how this plays out.”

In most scenarios, responsibility will attach first and foremost to the deploying organization, the enterprise that chose to implement the system. “Saying that we bought it from a vendor isn’t likely to be a defense,” she adds.

The underlying legal principle is familiar, even if the technology isn’t: liability follows the party best positioned to prevent harm. In an AI context, that tends to be the organization integrating the system into real-world decision-making, so what changes isn’t who’s accountable but how difficult it becomes to demonstrate appropriate safeguards were in place.

CIO as the system’s last line of defense

If legal accountability points to the enterprise, operational accountability often converges on the CIO. While CIOs don’t formally own AI in most organizations, they do own the systems, infrastructure, and data pipelines through which AI operates.

“Whether they like it or not, CIOs are now in the AI governance and risk oversight business,” says Chris Drumgoole, president of global infrastructure services at DXC Technology and former global CIO and CTO of GE.

The pattern is becoming familiar, and increasingly predictable. Business teams experiment with AI tools, often outside formal processes, and early results are promising. Adoption accelerates but controls lag. Then something breaks. “At that moment,” Drumgoole says, “everyone looks to the CIO first to fix it, then to explain how it happened.”

The dynamic is intensified by the rise of shadow AI. Unlike earlier forms of shadow IT, the risks here aren’t limited to cost or inefficiency. They extend to things like data leakage, regulatory exposure, and reputational damage.

“Everyone is an expert now,” Drumgoole says. “The tools are accessible, and the speed to proof of concept is measured in minutes.” For CIOs, this creates a structural asymmetry. They’re accountable for systems they don’t fully control, and increasingly for decisions they didn’t directly authorize.

In practice, that makes the CIO the enterprise’s last line of defense, not because governance models assign that role, but because operational reality does.

The illusion of distributed accountability

Most organizations, however, aren’t building governance structures around a single accountable executive. Instead, they’re constructing distributed models that reflect the cross-functional nature of AI.

Ojas Rege, SVP and GM of privacy and data governance at OneTrust, sees this distribution as unavoidable, but also potentially misleading. “AI governance spans legal, compliance, risk, IT, and the business,” he says. “No single function can manage it end to end.”

But that doesn’t mean accountability is shared in the same way. In Rege’s view, responsibility for outcomes remains firmly with the business. “You still keep the owners of the business accountable for the outcomes,” he says. “If those outcomes rely on AI systems, they have to figure out how to own that.”

In practice, however, governance is fragmented. Legal teams interpret regulatory exposure, risk and compliance define frameworks, and IT secures and operates systems. The result is a model in which responsibility appears distributed while accountability, when tested, is not — and it often compresses to a single point of failure. “AI doesn’t replace responsibility,” says Simon Elcham, co-founder and CAIO at payment fraud platform Trustpair. “It increases the number of points where things can go wrong.”

And those points are multiplying. Beyond traditional concerns such as security and privacy, enterprises must now manage algorithmic bias and discrimination, intellectual property infringement, trade secret exposure, and limited explainability of model outputs.

Each risk category may fall under a different function, but when they intersect, as they often do in AI systems, ownership becomes blurred. Mathews frames the issue more starkly in that accountability ultimately rests with whoever could have prevented the harm. The difficulty in AI systems is that multiple actors may plausibly claim, or deny, that role. So the result is a governance model that’s distributed by design, but not always coherent in execution.

The emergence and limits of the CAIO

To address this ambiguity, some organizations are beginning to formalize AI accountability through new leadership roles. The CAIO is one attempt to centralize oversight without constraining innovation.

At Hi Marley, the conversational platform for the P&C insurance industry, CTO Jonathan Tushman recently expanded his role to include CAIO responsibilities, formalizing what he describes as executive accountability for AI infrastructure and governance. In his view, effective AI governance depends on structured separation. “AI Ops owns how we build and run AI internally,” he says. “But AI in the product belongs to the CTO and product leadership, and compliance and legal act as independent checks and balances.”

The intention isn’t to eliminate tension, but to institutionalize it. “You need people pushing AI forward and people holding it back,” says Tushman. “The value is in that tension.”

This reflects a broader shift in enterprise governance away from centralized control and toward managed friction between competing priorities — speed versus safety, innovation versus compliance. Yet even this model has limits.

When disagreements inevitably arise, someone must decide whether to proceed, pause, or reverse course. “In most organizations, that decision escalates often to the CEO or CFO,” says Tushman.

The CAIO, in other words, may coordinate accountability. But ultimate responsibility still sits at the top and can’t be delegated.

The widening gap between deployment and governance

If organizational models for AI accountability are still evolving, the gap between deployment and governance is already widening. “Companies are deploying AI at production speed, but governing at committee speed,” Mathews says. “That’s where the risk lives.”

Consequences are beginning to surface as a result. Many organizations lack even a basic inventory of AI systems in use across the enterprise. Shadow AI further complicates visibility, as employees adopt tools independently, often without understanding the implications.

The risks are both immediate and systemic. Employees may input sensitive corporate data into public AI platforms, inadvertently exposing trade secrets. AI-generated content may infringe on copyrighted material, and decision systems may produce biased or discriminatory outcomes that trigger regulatory scrutiny.

At the same time, regulatory expectations are rising, even in the absence of clear legal precedent. That combination — rapid deployment, limited governance, and legal uncertainty — makes it likely that a small number of high-profile cases will shape the future of AI accountability, as Mathews describes.

Where the buck stops

For all the complexity surrounding AI governance, one pattern is becoming clear. Responsibility may be distributed, authority may be shared, and new roles may emerge to coordinate oversight, but accountability doesn’t remain diffused indefinitely.

When systems fail, or when regulators intervene, it often points at enterprise leadership, and, in operational terms, to the executives closest to the systems in question. AI may decentralize how decisions are made, obscure the pathways through which those decisions emerge, and challenge traditional notions of control, but what it doesn’t do is eliminate responsibility. If anything, it magnifies it.

AI accountability is a familiar problem, refracted through a more complex system. The difference is the system is moving faster, and the cost of getting it wrong is increasing.

The Trump administration is in early discussions about whether advanced AI models should be vetted before public release, according to reporting from the New York Times, the Wall Street Journal, and Axios.

The conversations center on systems capable of facilitating cyberattacks, particularly models that could help users identify and exploit software vulnerabilities. Officials are considering several options, including formal pre-release review processes and government-led testing for higher-risk systems. No proposal has been finalized, and no timeline has been set.

What has changed

The discussions mark a shift in tone, if not yet in policy. On Jan. 20, 2025, Donald Trump’s first day back in office during his second term, he revoked Biden’s Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence.

Three days later, he issued his own order, “Removing Barriers to American Leadership in Artificial Intelligence,” signaling a significant shift away from the Biden administration’s emphasis on oversight and risk mitigation toward a framework centered on deregulation and the promotion of AI innovation.

Among the things that order effectively ended: The Biden framework had introduced mandatory red-teaming for high-risk AI models, enhanced cybersecurity protocols, and monitoring requirements for AI used in critical infrastructure. The new discussions suggest certain security risks — particularly those tied to offensive cyber capabilities — warrant a more interventionist posture, even as the administration remains broadly opposed to sweeping AI regulation.

The Mythos factor

The discussion follows Anthropic’s recent introduction of Mythos, a model the company has described as representing a watershed moment for cybersecurity.

Anthropic has said Mythos Preview has found thousands of high-severity vulnerabilities, including some in every major operating system and web browser, and that AI models have reached a level of coding capability where they can surpass all but the most skilled humans at finding and exploiting software vulnerabilities. In one benchmark, the company reported significantly higher success rates compared to earlier models.

Anthropic has not released the model publicly. Instead, it launched Project Glasswing, committing up to $100 million in usage credits to a select group of technology and cybersecurity companies to use Mythos for defensive purposes — finding and patching vulnerabilities before malicious actors can exploit them.

Anthropic has also been briefing the Cybersecurity and Infrastructure Security Agency, the Commerce Department, and other stakeholders on the potential risks and benefits of Mythos Preview. OpenAI has developed a comparable model and has released it to a small set of companies through an existing trusted-access program.

What a review might mean

Pre-release evaluation of AI models is not a new idea, but it remains poorly defined in the US policy context. The Biden executive order Trump revoked had required developers of the largest AI systems to notify the government and share safety test results before deployment — one of several provisions the Trump administration characterized as burdensome obstacles to innovation.

The institutional picture has also shifted. The US AI Safety Institute, created under the Biden order to conduct pre-deployment evaluation and housed within the National Institute of Standards and Technology, was substantially reorganized after Trump took office. In June 2025, the agency was renamed the Center for AI Standards and Innovation, and its mission was revised.

Commerce Secretary Howard Lutnick framed the change as a repudiation of what he called the use of safety as a pretext for censorship and regulation. The renamed center’s mandate now includes leading unclassified evaluations of AI capabilities that may pose risks to national security, with a stated focus on demonstrable risks such as cybersecurity, biosecurity, and chemical weapons, potentially positioning it to play a role in any future review process.

Other governments have moved further and faster. The UK’s AI Security Institute has conducted pre-deployment evaluations of several frontier models, working directly with labs, including Anthropic and OpenAI, to assess risk thresholds before release. The EU AI Act, which began phasing in last year, establishes mandatory conformity assessments for high-risk AI applications.

The US has not established a comparable framework or legal authority to require such reviews.