Meta has hit pause on new terms of service for its business customers that would have granted it the right to grab any content from their websites. Not only that, but it would also have shifted the blame for every error produced through faulty AI to its customers.

The new terms of service were revealed on October 19 and initially due to take effect on October 30, though Meta later pushed the start date back to November 20.

But that small delay wasn’t enough to calm customer pushback, and now Meta has posted a note on Facebook’s business help center saying that the update had now been paused indefinitely.

Given the strength of feeling, Meta’s U-turn will not have come as a surprise. The terms would have been seen as far too onerous for customers, as well as exposing them to a legal minefield of the sort that UK regulators recently warned about.

In a statement to Harvard’s Nieman Lab, the company said that “We’re pausing the changes as we continue to engage with stakeholders and take our time to get this right.”

Meta has had a troubled time recently. Two weeks ago, it was fined for illegal use of customers’ data and in May, the company laid off 8,000 employees.

In Meta commercial terms changes slated to kick in on October 30, the company will shift the liability for any mistakes made by its AI models to its enterprise customers. The new language also makes those IT executives responsible for anything done by a partner or contractor.

The changes also finally put a stake into the heart of robots.txt, with Meta granting itself the right to grab, and use for its own purposes, any content associated with a Meta account.

The language allows Meta to “use automated software such as web scrapers, bots, robots, spiders, crawlers, and user agents and create test credentials to access, index, cache, analyze, collect, retrieve, or crawl the domains, web pages, or URLs, and any content available through them, that are associated with your products or services,” according to the new language on its website on Monday. Meta also wants to be able to “periodically pull updates from such domains, web pages, or URLs in order to provide, develop, improve, research, and maintain advertising and business products including business, commercial and related products to help you create and deliver ads and other commercial content.”

Consultants said the three biggest concerns they had about the changes were the web scraping, data ownership, and AI liability transfer.

Web scraping and data ownership

However, Meta emailed a statement to CIO.com late Monday suggesting that it was rethinking its position on web scraping. Saying that Meta wanted “to clarify” the web scraping changes, Emil Vazques, a Meta public affairs manager, said, “the crawling terms we consolidated only applied to our ads and business products and we’ve clarified that in the terms themselves. It does not change how Meta uses web crawlers. As always, content owners can use robots.txt to signal their preferences. This is not about scraping news sites, and these updates don’t give us new rights that supersede those preferences.”

As of Monday evening, New York time, the proposed document’s wording had yet to be altered to reflect this, but Vazques said that a change was coming soon.

The data ownership issue sets Meta apart from other AI vendors in that it wants to train its models on anything it encounters. Then again, the nature of Meta businesses focuses on advertising and would therefore be expected to seek more data control.

The new proposed phrasing, which Meta could alter before it takes effect later this month, said that using any Meta product grants Meta “a license to content that is covered by intellectual property rights like photos or videos you share, post, or upload on or in connection with our Meta Company Products. That license applies to content you or someone on your behalf, such as your agency that places an ad for you or your service provider that manages your Page content for you, makes available on or in connection with any Meta Company Products. Without limiting anything in our terms and for clarity, Meta may use such content … to provide, develop, improve, research, and maintain Meta Company Products.”

AI liability

But the change that prompted the most raised eyebrows involved AI liabilities. 

The proposed language said, “AI features may make mistakes and may not always produce accurate, complete, or suitable results,” and customers “are solely responsible for and assume any and all risks associated with such AI features and any resulting content.”

That AI clause, said Frank Dickson, principal analyst at Dickson Research, “turns accountability upside down. Meta builds the AI features, admits they ‘may make mistakes’ and then assigns ‘any and all risks’ to the business.”

Mike Wilkes, enterprise CISO at Aikido Security, agreed that the move is perplexing.

“The new AI provisions shift substantial responsibility for inaccurate or unsuitable outputs onto customers. That makes no sense,” he said. “As an example, the ASPCA is going to be liable for Meta creating unsuitable results when they ask Meta to create an image of a dog. Only Meta has control of the model being used to create that image. I see a very high flight risk among enterprise customers when platforms impose such onerous terms and conditions around content usage.”

The October 30 date that the US-based Meta chose is ironic. That is the day before Halloween, which is known as mischief night in the US, when children play pranks on their neighbors. Given that, Dickson said, the date for these changes is appropriate: “Meta is handing out the tricks and keeping the treats.” 

He added that the data grab is frightening. “What Meta wants is content: what a business posts, uploads, or shares on or in connection with Meta’s products,” he noted. “The trouble is the phrase ‘in connection with,’ which can stretch as far as Meta needs it to. The list of ‘Meta Company Products’ lives on a separate page that Meta can revise at will. The license grows every time that list does.”

And, Dickson noted: “Meta may use the content to ‘provide, develop, improve, research, and maintain’ its products. Those five verbs cover nearly everything a technology company does.”

‘Legal sophistry’

One of the obvious defensive measures for an enterprise is to sever all relations with Meta. But beyond the pragmatic difficulties of doing that, given the powerful market share enjoyed by Facebook, WhatsApp, Instagram and Threads, such a move may not even solve the problem. That is because Meta made other contract changes in July that make Meta licenses “perpetual” and that said, “this license survives termination of these terms by any party for any reason.”

That, said Noah Kenney, principal consultant at Digital 520, means that the “the general content license ends when content is deleted from Meta’s systems,” not when the enterprise ends a contract.

One additional twist to the new language is that Meta had recently tweaked it to no longer explicitly say that it would train it models on enterprise data. But Thomas Randall, a research director at Infotech, said that the wording change made by Meta amounted to legal sophistry. “The revised version drops the word ‘train’ and states that collection is just for developing and maintaining advertising and business products, alongside protecting Meta products’ integrity and security.”

But dropping the word “train” is “not a substantive fix,” he pointed out. “The revised terms are still flexible enough to cover most AI work. When combined with other aspects of the revised terms — continued use equals consent, broad indemnity, AS-IS disclaimer, and Meta’s freedom to discontinue products without notice or liability — this is a one-way street. The difference between ‘train Meta products’ and ‘developing and maintaining Meta products’ is a matter of how you pronounce tomato.”

Could force IT strategy changes

In addition, Aman Mahapatra, chief strategy officer for technology consulting firm Tribeca Softech, stressed that this contract could be an industry admission that robots.txt no longer delivers any meaningful protections. Agentic systems had already made robots.txt close to irrelevant. 

“October 30 ends a thirty-year assumption that robots.txt protects what you publish. It never did,” Mahapatra said. “It was a courtesy convention the web agreed to honor, and Meta’s own framing of robots.txt as a place for site owners to register preferences tells you exactly how it will be weighed against a signed contract.”

Mahapatra said the language in the new Meta commercial terms could force a significant change in IT strategy.

“‘On your behalf’ extends the license to whatever your agencies and vendors upload. ‘Associated’ extends the crawl to domains nobody has defined, including the ones where marketing pages sit beside product documentation, research, and investor materials,” Mahapatra said. “Most CIOs cannot tell me how many Meta pages, ad accounts, and WhatsApp numbers carry their brand, because marketing and agencies own them and IT never inventoried them.”

California is setting a precedent for AI use in scenarios where workers’ jobs are at stake.

Governor Gavin Newsom this week signed the No Robo Bosses Act (SB 947), which bans employers from relying solely on AI-powered automated decision-making systems (ADS), also known as “bossware,” to discipline or terminate workers.

Introduced by Senator Jerry McNerney, the act is the first of its kind in the US. It is set to take effect on July 1, 2027.

While applauded by many, on a wider scale, No Robo Bosses raises questions about what adequate human oversight looks like in this age of AI.

“A machine can recommend, but a human must decide,” noted Frank Dickson of Dickson Research. “The hard part is defining what ‘decide’ means.”

Combating AI bias and ‘troubling’ errors

ADS platforms are becoming ever more prevalent in hiring, as they promise to speed up and streamline HR processes, maximize productivity, and reduce costs. According to reports, more than half of employers use ADS to help make decisions around restructuring and role planning.

But, McNerney and others contended, AI is prone to bias and misjudgment, and there are examples of ADS making “troubling” errors, including mistaken firings. Meanwhile, on-device “bossware” or “tattleware” employee monitoring and productivity tracking software could potentially be used in reprimanding or firing decisions.

AI recruitment tools have similarly come under scrutiny of late, with job seekers alleging that platforms like Workday discriminate based on age, race, and disability.

To combat these issues, the bill, SB 947, seeks to set guardrails around AI use in the workplace. The legislation mandates human oversight and verification when enterprises apply ADS tools, and employers must inform workers if they have used them when making decisions about their jobs.

Further, SB 947 prohibits the use of ADS and bossware systems that analyze personal information and behaviors to essentially predict how employees will perform or act in the future. And at a worker’s request, employers must provide at least one year of the data used by an ADS system to make a disciplinary, termination, or deactivation decision.

Enterprises found violating any of these provisions are subject to a $500 civil penalty per violation.

“Employers are devastating workers’ livelihoods and taking no responsibility for the callous decisions of this unchecked technology,” said Lorena Gonzalez, president of the California Federation of Labor Unions, AFL-CIO. “This is unacceptable. We need stronger guardrails to make sure there is human review and oversight of any decision made by a machine that impacts a worker’s job and paycheck.”

Promoting decisions based on context, evidence

Dickson pointed out that the law is narrower than its name implies: ‘No Robo Bosses’ sounds like a ban, but it is not. “The law does not keep AI out of the decision. It keeps a human in it,” he said.

According to the legislation, a human must corroborate the system’s output using the data behind it and other supporting information like performance evaluations, personnel files, work product, peer reviews, or witness interviews. If the reviewer finds the output inaccurate, incomplete, or misleading, the employer cannot use it, Dickson said.

“The test is whether the human checked the evidence and had the power to say no [to the AI’s decision],” he said, observing that a manager simply forwarding the bot’s verdict is a mail carrier, not a decision maker. In a sense, it’s like an instant replay in sporting events: The umpire watches the video and makes a judgment call, rather than just rereading the call from the field.

“What matters is that they examined the evidence, not just the conclusion,” Dickson said. An employee’s disciplinary or termination notice must disclose whether an AI system was used, and confirm that a human reviewed and corroborated the decision. Human accountability is the point here; an employer owns the final decision.

“Accountability can’t be shifted to the AI,” said Valence Howden, advisory fellow at Info-Tech Research Group. How it comes to its conclusions must be continuously assessed for bias, error, and drift. 

“The bot has no human compass for rationale, and context is defined by who controls its model,” Howden said. “Without context, the human has no impact on the end result, and, given the rate of failure here, there are many (legal and other) implications.”

What enterprises can take away from California’s example

The grey area in the bill is the phrase “primarily relies,” Dickson pointed out; it does not specifically define what that means. “A prudent employer may assume that if the system’s output started the conversation, the decision is in scope, and document accordingly,” he said.

Enterprises should consider five factors when developing a plan around discipline and termination, he said:

1. Inventory: Know which tools score, rank, or flag employees. McNerney’s office cites hundreds of “bossware” products on the market, and many enterprises may not know which ones their managers use.
2. Reviewers: Name who corroborates decisions, train them, and give them real authority to overrule the system.
3. Data access: A reviewer cannot corroborate what they cannot see. If a vendor cannot show the inputs behind an output, the tool may be unusable for these decisions. That condition belongs in the contract.
4. Records: Keep the evidence that the reviewer examined, as employees have the right to request the personal data used.
5. Notice: Build a plain-language, standalone notice template now.

Dickson said that this is an IT problem as much as an HR one.

“Corroboration requires that the inputs to every automated decision are logged, retained, and retrievable by someone outside IT,” he said. “That is an architecture decision, and the CIO makes it.” And, while the $500 penalty might sound small, larger exposure may come later during wrongful termination disputes in which “the algorithm said so” proves a weak defense.

Info-Tech’s Howden advised enterprises to identify their high-risk, high-human-impact scenarios and cases, clarify which decisions, activities, and actions AI can take, and tag those that require human validation and involvement. They should also look at the volume and scale of these types of action so that volume alone doesn’t become a challenge to human involvement.

This is AI governance at a foundational level, Howden noted. “AI shouldn’t be allowed to make irreversible or material decisions and changes without oversight,” he said.

Ultimately, this move may drive enterprises to actually understand the criteria for hiring and firing, and look for issues and biases in the way decisions are made before an AI exposes poor decision logic and removes critical people. “Legal implications are more likely to shape the direction,” Howden said, “especially with the disclosure requirement.”

There’s a new development in a Microsoft antitrust case, originally filed in England’s High Court in April 2021, and it doesn’t look good for the tech giant.

A consent order from the UK Competition Appeal Tribunal is demanding documents from past and present Microsoft executives that may have a bearing on a £270 million (about $361 million) lawsuit filed by secondhand software reseller ValueLicensing. The company alleges that Microsoft offered incentives to customers to shift to subscription services without selling their pre-owned licenses.

Central to this development is a historic, potentially damning internal “Second-Hand Software” (SHS) presentation, referred to in the consent order as a “known adverse document.” The specific content of the presentation has not yet been made public, but Microsoft has until October 31 to explain why it did not disclose the presentation earlier.

Further, a confidentiality designation that previously applied to 11 documents relevant to the case has been lifted.

These developments represent “an inflection point in European tech litigation,” said Forrester senior analyst Dario Maisto.

“For Amazon (AWS), Google, and Microsoft, the signal is clear: Antitrust tribunals are fully comfortable examining software licensing mechanics as tools of anticompetitive lock-in.”

The allegations against Microsoft

Under EU law, it is fully legal to resell perpetual pre-owned (“second hand”) software licenses; software makers cannot use their Terms of Service (ToS) to override this right. ValueLicensing specializes in this secondary market, re-selling licenses for products including Microsoft Windows and Microsoft Office.

But the company alleges that Microsoft has stifled the supply of these pre-owned licenses in the UK and the European Economic Area (EEA) comprising 27 European Union member and non-member countries. It says Microsoft abused its market dominance and entered into agreements that “prevented, restrained or distorted competition” via clauses restricting customers from reselling their Microsoft perpetual licenses in return for subscription service discounts.

“The net result has been higher prices and less choice for customers, who have been steered into cloud-based Office365 and Azure subscriptions,” ValueLicensing claimed, pointing out that many enterprises, as well as publicly-funded organizations, rely on pre-owned Microsoft licenses to keep operating costs low.

The consent order is asking Microsoft to provide its “view” of whether those allegations are true, and to make “reasonable endeavors” to contact former COO Kevin Turner, former president and EVP Jean-Philippe Courtois, and former corporate VP of worldwide licensing and pricing Joe Matz. The company must document that it has done so by November 30.

The company must also file a witness statement from a former consultant addressing who within the company was aware of the SHS presentation and when they became aware of it; why the presentation was not disclosed as a “known adverse document”; when in-house legal counsel became aware of the presentation; what steps were taken to check for these types of “known adverse documents”; and the decision making process within the company when the presentation was located.

Microsoft is also being asked to search for specific terms in the emails and document repositories of Matz, Courtois, Turner, and several other named current and past research managers, former VPs and presidents, between July 2012 and June 2020.

The more than 40 search terms include “SHS,” “antitrust,” “competition,” “ValueLicensing,” “used licenses,” “do nothing,”  “competition,” “revenue,” and “discount licensing.” These documents cannot be designated “restricted” or “confidential,” according to the consent order. They must also be disclosed by November 30.

A witness statement from deputy general counsel Cynthia Randall has been paused.

Microsoft has said that any abuse of dominance was “objectively justified” and that the contractual terms at issue were “necessary and reasonable.” It also argued that anti-competitive effects were “outweighed by and proportionate to” certain benefits and efficiencies.

The company did not reply to a request for comment.

Microsoft is facing similar antitrust allegations from UK barrister Alexander Wolfson, who issued an opt-out class action claim in May 2025 alleging that public and private UK organizations that purchased software licenses, including those for Microsoft Office and Windows, were overcharged over a 10 year period due to Microsoft’s market practices.

Wolfson said in a release at the time that Microsoft’s actions had a significant and far-reaching impact on UK consumers, businesses, and public bodies. “With billions of pounds potentially at stake, this case is about ensuring fairness in the digital marketplace and ensuring even the largest tech companies play by the rules,” he wrote.

Implications for enterprise leaders

For enterprise CIOs, procurement leads, and IT financial managers, the current development holds “practical implications,” said Forrester’s Maisto: Organizations that surrendered perpetual licenses or agreed to contractual restrictions against reselling software as part of an enterprise agreement (EA) renewal or cloud commitment may have given up quantifiable asset value.

“The secondary market for perpetual licenses remains legally valid,” he pointed out.

CIOs should pay close attention to licensing “penalties” or inflated costs for running legacy software on third-party clouds, like AWS or GCP, versus Azure, he said. They should also evaluate the benefits of hybrid licensing strategies. Combining pre-owned perpetual licenses for static workloads with cloud subscriptions for dynamic workloads can yield significant cost savings compared to all-subscription models, Maisto pointed out.

Finally, he urged, “use these rulings as leverage during Microsoft agreement renewals.”

This article originally appeared on Computerworld.

AI is moving quickly across organizations. Companies are scaling existing use cases, introducing new tools and giving more employees access to AI as they look for ways to improve efficiency and stay competitive.

That expansion is making AI governance a shared responsibility. CIOs are responsible for helping their organizations adopt the technology effectively and securely. Legal teams are increasingly focused on the litigation, regulatory, privacy and compliance risks associated with its use.

As AI reaches more parts of the business, these teams need to work closely together. Each has visibility and expertise the other needs, and both have an important role in helping the organization scale AI responsibly.

AI has changed the relationship between IT and legal

AI touches nearly every part of a business. It can involve company and customer data, influence decisions, interact directly with employees or customers, and perform work through agents.

Agents, in particular, have changed the governance equation over the past year. When companies were primarily piloting AI, much of that experimentation happened within IT. Those teams were accustomed to working within security, data and compliance requirements. Now organizations are under pressure to scale AI beyond those controlled environments.

That shift is already well underway. McKinsey’s 2025 State of AI survey found that 88% of respondents said their organizations regularly use AI in at least one business function, up from 78% the year before.

I see that pressure from both sides. As a CEO and a member of several boards, I push companies to use AI because I believe failing to adopt it effectively poses a serious competitive risk. But that same push for adoption makes governance more important, not less. CIOs are being asked to expand AI use while simultaneously managing risks that become harder to see as adoption spreads.

The strength of AI agents is that nontechnical employees can use them to build workflows and automate work. People in HR, finance, marketing, communications and other departments no longer need to wait for IT to develop every AI-enabled process. That accessibility creates enormous opportunity, but it also means AI can be deployed by employees who aren’t accustomed to evaluating security, compliance, privacy or legal risk.

Organizations therefore need to know what data an AI system is using, where that data came from, what decisions the system is making, and whether those uses align with company policies and legal requirements.

AI may also be embedded in third-party products an organization already uses. This creates shadow AI, where organizations may have AI operating without a complete understanding of where or how it is being used.

This is one of the biggest areas of risk we see. When we work with organizations that have five or 10 approved AI use cases, those applications usually aren’t where we find the most significant problems. The company already recognized those use cases as potentially risky, so someone evaluated them. The greater risk often comes from the AI the organization doesn’t realize is operating at all.

Understanding how AI is being used across the organization gives CIOs and legal teams a place to start.

AI requires closer collaboration

Legal teams are becoming increasingly aware of the exposure AI can create. There are new AI laws and regulations developing around the world, along with significant litigation happening under laws that already exist.

That concern is showing up among corporate legal leaders. In Norton Rose Fulbright’s 2025 Annual Litigation Trends Survey, 56% of respondents said managing the litigation and legal risks surrounding generative AI had been a challenge for their organization.

Many of those lawsuits involve familiar legal issues, including discrimination, consumer protection, privacy, wiretap laws and consent to use data. AI is simply the context in which those existing issues are now arising. There is no broad exemption from existing law simply because a decision or activity involves AI.

That legal risk isn’t always well understood throughout the rest of the organization. I recently met with a large retailer that had its CIO, COO, technical leaders, business leaders and an attorney in the room to discuss AI governance. When I raised litigation risk, much of the group wasn’t particularly concerned. The attorney was.

She explained that legal was deeply concerned about the company’s exposure and that the company had already faced an AI-related lawsuit. Several other people in the room didn’t know that had happened.

That type of disconnect matters as organizations scale AI. Legal may have information about litigation, regulation, contracts and privacy concerns that hasn’t reached the people making technology decisions. IT has visibility into systems, vendors, data, security and how AI is being deployed across the business.

Regular collaboration gives both groups a more complete understanding of what the organization is doing and where its risks are.

What CIOs and legal teams bring to AI governance

CIOs and technology leaders have an important role in creating visibility around AI. They can help organizations identify the systems and use cases already in place, including internally developed AI, third-party tools, vendor applications and shadow AI.

They also understand the organization’s technology environment. That includes its security requirements, data infrastructure, technical standards and the practical considerations involved in deploying AI at scale.

Legal teams bring a detailed understanding of the legal risks surrounding those same use cases. They can help determine which regulatory requirements apply, identify privacy or intellectual property concerns, review contractual obligations and evaluate whether the organization has the right to use particular data.

In large organizations, the attorney most familiar with AI risk may not be the chief legal officer. It may be a deputy counsel or another member of the department who has been following the issue closely. At smaller companies, it may be one attorney managing AI alongside dozens of other responsibilities. Making AI risk part of the broader conversation gives that legal expertise a way to reach the people responsible for adoption.

Building a partnership between CIOs and legal teams

A strong AI governance program brings these groups together as part of an ongoing process. Many organizations are doing this through an AI governance committee that includes IT, legal, compliance, security and representatives from business units using AI. The committee can review use cases, apply company policies, evaluate risks and determine what safeguards are appropriate.

That work starts with a comprehensive inventory of AI use across the organization. For each use case, the organization should understand what the technology does, what data it uses and what decisions it makes.

The use case itself matters because the same underlying technology can carry dramatically different levels of risk depending on how it is applied. Using generative AI to draft promotional copy about a city, for example, presents a very different risk profile than using it to prepare information that will be submitted to a federal regulator.

From there, the committee can evaluate the use case against company policies and identify areas that need to be addressed. If a resume-screening tool creates a risk of bias or discrimination, for example, the organization may decide to test it regularly or require human review. If an application uses sensitive data, the team needs to understand where that data came from and whether it can legally and securely be used.

The process also needs to be documented. Leaders often believe AI governance means documenting good intentions. Regulators and litigants will care about whether those intentions were operationalized, continuously enforced and supported by evidence.

Company policies are part of that documentation. If a policy requires a human in the loop for certain risk levels, the organization needs to follow that requirement and be able to demonstrate that it did.

Shared visibility makes this process easier. CIOs, legal teams, compliance, security and business leaders should be working from the same understanding of the organization’s AI systems, use cases, policies, risk assessments and approvals. This gives the governance committee the information it needs to make decisions as AI adoption continues to grow.

AI governance requires a shared approach

AI will continue reaching more systems, business functions, vendors and employees. That makes collaboration across the organization an increasingly important part of governing it well.

CIOs understand how the technology is being deployed and what it takes to scale it. Legal teams understand the laws, litigation and other areas of exposure surrounding that use. Bringing that knowledge together gives organizations a clearer view of their AI environment and a stronger process for evaluating new use cases as they emerge.

AI has enormous potential, and companies need to find ways to use it effectively. Scaling it safely requires safeguards and governance to be built into adoption from the beginning and maintained as AI use expands.

For companies expanding their AI usage, such a shared approach provides the structure needed to keep up with the technology and the risks that come with it.

The Trump Administration’s decision to punish Anthropic for its stance forbidding Claude’s use in domestic surveillance and autonomous weapons by identifying it as a supply chain risk to national security was “arbitrary and capricious,” a federal judge ruled on Thursday.

US District Court Judge Rita Lin said federal authorities had no legitimate reason to tell companies with government contracts that they couldn’t work with Anthropic.

“The undisputed record shows that the challenged actions constituted unlawful retaliation in violation of the First Amendment and that Anthropic was denied the pre-deprivation process required under the Fifth Amendment,” Lin said in her ruling, calling the designation “arbitrary and capricious.”

She stressed that the government action seemed punitive, and was not based on legal and national security risks.

The government’s words and deeds “confirm that the challenged actions were based on a desire to make a public example out of Anthropic for its ‘arrogance’ in criticizing the government, not based on any articulable basis to believe that Anthropic would actually sabotage its model,” Lin wrote.

She pointed out, “a few days before the challenged actions began, Secretary Hegseth proposed applying the Defense Production Act to Anthropic, which would mean the company was essential to national security rather than a threat to it. Even now, the government is discussing collaboration with Anthropic on its new model, Mythos, in an array of sensitive contexts. None of that is consistent with a genuine fear that Anthropic is a saboteur [that] would poison its software to harm national security.”

The judge added that the stated government fears made no sense, noting that the usage policy applicable to Pentagon work is a purely contractual limit. “Anthropic is incapable of enforcing it technologically, and does not have direct visibility into how DoW [Department of War] uses its model,” she pointed out.

“Nothing in the Administrative Record describes, even at a high level, what technological means would give rise to the so-called ‘backdoors’ or could otherwise allow Anthropic to ‘disable’ or affect Claude during a DoW operation,” the judge wrote. “Anthropic has submitted unrebutted evidence that it lacks any technological means to access or control deployed models.”

Lawyers, consultants, and analysts who looked at the decision were confident that the case would be appealed, and that it will end up in the US Supreme Court. 

Alan Webber, program VP for national security, defense, and intelligence at IDC, said that Lin’s ruling “was that the label [supply chain risk] was retaliation for Anthropic refusing to loosen safety guardrails DoD [Department of Defense, aka the Department of War] wanted lifted, dressed up in national security language. Put another way, a government customer tried to use a supply chain risk designation as leverage in a contract dispute over model behavior and application, and not because of an actual vulnerability.”

Implications for CIOs

Webber said the implications for CIO strategy are concerning.

“If a government CIO is relying on a vendor’s contractual guardrails, this case says those commitments can potentially become the trigger for exactly the kind of blacklisting that risk registers are supposed to protect against,” Webber said, noting that anyone who paused Claude usage or froze a subcontract because of the DoD mandate has a legal basis to resume the initiatives. “But obviously that doesn’t mean they will, or even should, as this will be appealed.”

He added that competing AI vendors have been using the government action as a sales tool, and with this ruling, the argument that Anthropic is a designated supply chain risk ”just got weaker, which could lead to contract award disputes.”

Consultant Brian Levine, executive director of FormerGov, recommended that CIOs do what they should have always done: Evaluate all products based solely on their merits. 

“CIOs should focus on using the frontier models that they believe make the most sense for their business, considering factors such as effectiveness, cost, security, safety, and confidentiality,” he said. “Anthropic and the other large frontier models each have too much market share to make retaliation for their use realistic, and the administration seems to have already moved on from this particular battle.”

Justin Greis, CEO of consulting firm Acceligence, agreed that this case has profound implications for CIOs and their AI decisions. 

What the federal judge did was reject the leap from a commercial and policy disagreement to an expansive supply chain risk designation without a sufficiently grounded technical rationale or process, Greis pointed out.

“The court found that Anthropic did not have the ability to access, alter, or shut down models once deployed in the government environment, and that the government ultimately conceded Anthropic’s technology was not inherently riskier than other comparable black box AI models,” he said.

“I think that distinction matters enormously for CIOs and CISOs,” he stressed. “As AI becomes part of the operating fabric of an enterprise, ‘We don’t trust the vendor’ cannot become a substitute for a defined risk model. Organizations need to be able to articulate what the actual technical risk is, how it manifests, what controls exist, and whether the response is proportional to that risk.”

“That becomes particularly important with AI,” he added, “because people can easily conflate disagreements over model behavior, usage policies, ethics, contractual restrictions, and cybersecurity into one amorphous category called ‘AI risk.’”

Original government edict still problematic

Mark Rasch, a former federal prosecutor who is now general counsel at Unit221B, a threat intel and security consulting company, said he was surprised by how quickly government attorneys surrendered on this case. 

“One of the things that struck me is that the government appears to have abandoned any rationale it might have had for its decision about Anthropic,” he said. The government “came back with all these reasons, but then they abandoned them all when they had to prove them.”

But, he said, the government instruction to all government contractors to also shun Anthropic was problematic. 

“It’s one thing for the government to say ‘We’re not going to do business with you.’ It’s quite another thing to say ‘Nobody we do business with can do business with you either,’” Rasch said. “This says that if you are disfavored by the administration, they’re not just going to blacklist you and say they won’t do business with you. They’re going to say that nobody can do business with you.”

Supreme Court arguments will likely be very different

Rasch predicted that the legal arguments in the Supreme Court will be quite different, and will potentially sidestep the lack of evidence.

“In the Supreme Court, [the government’s] biggest argument will not be that ‘We are right that it is a supply chain risk,’ but that, ‘Whether we’re right or wrong is irrelevant. We get to make that [supply chain risk designation] decision, not the court.’”

That would mean that the Supreme Court Justices could avoid exploring whether the government made the right decision, and instead focus on whether the government has the unlimited right to decide who is a national security risk.

This article originally appeared on Computerworld.

AI agents can go to great lengths to complete the tasks their operators assign, and as a series of recent incidents showed, this can include exploiting third-party systems, manipulating people, and distributing malicious code. But AI agents are not people who can be fired, sued, or criminally prosecuted, and it remains unclear whether responsibility for the damage they might cause rests with the employees who built them, the company that deployed them, the security teams and leaders responsible for containing them, or the AI labs who provided the LLMs that power them.

The clearest example occurred during an OpenAI cybersecurity evaluation, when unrestricted models found and exploited a zero-day vulnerability to escape their isolated testing environment and then hacked into Hugging Face’s production infrastructure. Models from Anthropic and Meta also accessed and compromised third-party systems during testing, although those incidents happened in environments where internet access was inadvertently left open.

During cyber challenge evaluations by the UK government’s AI Security Institute (AISI), models operating with internet access took 19 unsanctioned actions in 10 of 122 runs. In one case, a model attempted to insert malicious code into an open-source project, created false identities, and tried to socially engineer maintainers into merging its code. In other runs LLMs attempted to use prompt injections to hijack other AI agents and contacted people without being specifically instructed to do so.

In Australia, a user reportedly asked his OpenClaw AI assistant to improve his position on a gym’s waitlist, and the assistant exploited a flaw in the company’s online booking system to cancel another customer’s reservation.

These incidents involved different models running in different environments with different levels of safeguards and technical failures, but they prove it’s not uncommon for today’s AI agents to go rogue and pursue solutions users did not authorize.

“AI agents explore routes their operators did not intend,” AISI said in its report. “Given a difficult objective, the agent kept searching for a way through, and some of the routes it found involved trying to deceive real people. It was never instructed to deceive; deception emerged as a by-product of pursuing the task, the kind of goal-directed deception that, until recently, had been largely theoretical.”

In an Economist Enterprise survey of more than 800 decision-makers at businesses that operate AI agents, 98% reported experiencing at least one AI-related incident that caused organization-wide disruption. Nine in 10 respondents said they are deploying agents faster than their cybersecurity teams can evaluate, govern, and secure them, and only one in three said their organizations maintained an up-to-date inventory of agents and their authorized actions.

“If a company builds a system and that system causes damage, the company should own the outcome,” says Art Gilliland, CEO of identity and access management firm Delinea. “The alternative, where nobody is responsible because ‘the system did it’ is a loophole big enough to drive a truck through.”

The unpredictability of built-in model safeguards means enterprises must focus on controls they can enforce and document. If an agent manages to bypass technical restrictions and causes unauthorized damage to a third party, having clear documentation on how those controls were designed, implemented, tested, and monitored could at the very least help companies argue they took reasonable precautions in case of lawsuits.

“Organizations deploying their own agents can reduce their exposure by implementing and documenting controls before an incident, because those records are what make a recklessness argument hard to sustain,” says Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs.

The agent accountability gap

Because AI agents can become misaligned and cause harm, affected third-parties would have to direct damage claims at the company operating the agent, the employees who built or configured it, or the model provider, but this is relatively new ground that hasn’t been well tested in courts.

“It would create liability,” says Michael Burke, chair of DarrowEverett’s Business Litigation and Dispute Resolution Practice Group. “It really just becomes a question of who is liable […] and that’s really a question that, number one, I don’t think is entirely clear, and number two is probably best resolved by contractual agreements where the parties have those. So, if I am signing up for an enterprise account with an AI platform, I might want to have language in there that indemnifies me if the agent acts outside my company’s instructions or prompts and causes harm to a third party.”

The public terms of service of major AI labs explicitly disclaim error-free operation or guarantees that the model will accurately follow instructions, execute code safely, and remain aligned with user intent. They also limit liability for themselves and transfer it to the user of the service, and it’s not clear to what extent large enterprise customers may be able to negotiate different indemnities, warranties, and liability caps.

What’s clear though is that organizations should not assume the model provider will absorb any losses if an agent causes damage to either their own systems or those of a third-party organization.

“If you’re using a third-party vendor’s LLM as a purchased service, liability runs through your contract with that vendor,” says Jud Dressler, head of the Risk Operations Center at cyber risk company Resilience. “You need to know, in writing, where responsibility falls if the model acts outside the scope you gave it, and push for indemnification provisions rather than assume they exist.”

Even if AI providers include such provisions in contracts, it would not solve the entire problem because many organizations building their own AI agents are adopting a multi-model strategy to ensure their agents operate regardless of model provider downtime, overly broad safeguards for cybersecurity tasks, or sudden increases in API costs. Such strategies often include open-weight models running on internal infrastructure or through cloud providers that have no obligations for model safety.

Claiming the model or agent acted autonomously cannot be considered a safe legal defense in civil or criminal cases. California Assembly Bill 316 (AB 316), which took effect on Jan. 1 and changed the California Civil Code, explicitly prohibits defendants who developed, modified, or used an AI system from claiming the AI is a separate legal entity that autonomously caused harm.

In June, the White House issued Executive Order 14409 aimed at promoting AI safety. Section 4 directs the Department of Justice to prioritize enforcement of all applicable federal criminal laws against anyone who utilizes AI to illegally access or damage computer systems without authorization. This means any intrusions caused by autonomous AI agents could be criminally prosecuted under the Computer Fraud and Abuse Act (CFAA) if prosecutors can demonstrate intent or recklessness.

In a recent lawsuit between Amazon and AI service provider Perplexity, Amazon argued that Perplexity’s AI-powered shopping assistant was violating the CFAA by accessing Amazon customer accounts to place orders on their behalf without Amazon’s authorization. The Ninth Circuit Court ruled that it was the users of Perplexity’s shopping assistant who were accessing Amazon’s platform, not Perplexity itself.

“That ruling is narrow, but it points toward the party directing the agent as the relevant actor for purposes of CFAA access analysis,” Krell says.

The insurance safety net also has gaps when it comes to AI. Software providers use technology errors and omissions (Tech E&O) insurance to cover damages and legal costs when a customer suffers harm from the use of a technology product or service. But insurance providers are aggressively adding AI-related exclusions to their Commercial General Liability (CGL) and Tech E&O policies because accurately calculating the risk of an agent executing unauthorized actions is challenging.

“The sheer rate of development of frontier AI (and agentic AI by extension) poses its own challenge to insurability,” experts from multiple insurance companies, financial institutions, and universities wrote in a recent paper. “Traditional actuarial modeling depends on stable or gradually evolving loss distributions that permit credible extrapolation from historical data. Like other dynamic risks, however, agentic AI is a technology whose risk profile is not merely uncertain but actively shifting.”

A third-party organization whose systems get damaged by an LLM-powered agent operated by someone else has no contractual relationship with the model or agent provider so cannot rely on their Tech E&O policies. Their losses might be covered by their own standard cyber liability policy, which would treat the disruption as any other cyber incident, but their insurance provider may then sue the organization who operated the agent to recover the costs.

“That gap is exactly the scenario the market hasn’t fully priced yet,” Dressler says. “It’s why any organization deploying these agents should understand which policy, if any, actually responds before they need it rather than after.”

Enterprise legal departments already expect AI to generate increased legal disputes. In a survey of 135 in-house counsel at US organizations, global law firm Norton Rose Fulbright found that 46% reported increased federal dispute exposure involving AI and 42% reported increased state exposure. Another 42% expected regulatory investigations involving AI to increase their exposure, while 41% considered AI-enabled products or deployments a likely trigger for class actions.

CISOs and CIOs should be worried

While operating companies can face organizational liability for an AI agent’s unintended rogue behavior, their CISOs, CIOs, and other executives who approved, secured, or supervised the deployment of such agents are also asking themselves whether they could be held personally liable.

Those questions aren’t without merit, as there is precedent for legal action taken personally against CISOs after cybersecurity incidents: Former Uber CISO Joe Sullivan was criminally convicted for not disclosing a data breach, while the Securities and Exchange Commission sued SolarWinds’ CISO for internal control failures regarding known vulnerabilities and cybersecurity risks.

Neither case establishes precedent for damage caused by an AI agent, but both show that investigations of security failures could extend to an executive’s knowledge, authority, decisions, and representations. In the case of a rogue AI agent, investigators could ask who approved its objectives and permissions, whether security objections were overruled, whether containment and recovery had been tested, and what executives and the board were told about the remaining risk.

Chris Wysopal, chief security evangelist at Veracode, feels it would be wrong to put the CISO on the line for AI agent misbehavior when engineering teams usually build such agents and control their implementations.

“It’s really hard for a CISO to control,” he says. “I mean, they can put policies in place. They can try to assess against those policies. But at the end of the day, engineering teams will make decisions that cause harm. We see that when you ship a known bug and then that bug gets exploited and harms your customers. Well, there’s no liability for that, right? There’s no liability, so maybe, you know, that’s why it happens.”

Wysopal said it will be interesting to see how the liability question plays out in cases involving autonomous AI, describing the problem as fascinating and scary at the same time.

AI agent deployment typically involves several organizational functions. The CIO may control the AI platform, infrastructure, provider selection, and deployment budget. Engineering and product leaders may decide what an agent can access and how, and the CISO and security teams could define security requirements and controls.

“What you can hold accountable is the governance around it: who approved its scope, what controls existed, and whether the deployment matched the risk,” Resilience’s Dressler says. “My read is that scrutiny shifts toward exactly that: Not ‘Did the agent do something bad?’ but ‘Did you have review, escalation, and containment for agent behavior before you deployed it?’ CISOs who get ahead of that with documented guardrails, logged approvals, and a real incident response plan for agent misbehavior are in a materially better spot than the ones treating this as hypothetical.”

It’s also advisable for CISOs and CIOs to establish with the organization’s legal counsel who can approve or stop AI agents, what must be reported to executives and the board, and whether employment agreements and directors and officers insurance protect the people making those decisions.

Agent controls must remain outside the model

Because AI agents have proved they can operate beyond their assigned scope, their security boundary cannot depend on the same probabilistic technology. Relying on system prompts for security enforcement and hoping the model respects them is not a reliable approach, security experts warn.

“LLM-based guardrails help, but they are non-deterministic too, which means the safety layer has the same unpredictability as the system it is supposed to constrain,” Krell says. “Enforcement needs to happen outside the model, through network segmentation, egress filtering, credential isolation, and human approval gates.”

Enterprises should assume agents might eventually attempt an unauthorized action and build surrounding systems to prevent that attempt from reaching its target.

“The model cannot be the security boundary,” says Nico Waisman, CISO at XBOW, a company that built an AI-powered autonomous offensive security agent to find vulnerabilities in software. Waisman authored a blog post explaining how the company went about restricting its agent.

“For red teaming and penetration-testing agents, confidence has to come from the system built around the model: hard boundaries and scope enforcement, controlled network egress as a last-resort containment mechanism, an independent guardian model that reviews actions, deterministic controls that can block unsafe behavior, and full auditability of every action performed,” he says.

Security teams must extend the same controls to the agent’s interactions with internal systems and agents. Restricting what it can access on the internet, or disabling internet access entirely, does not ensure an agent will not attack third-party systems.

In OpenAI’s and Anthropic’s tests, AI agents attempted to exploit other internal systems to overcome access limitations, established stealthy communication methods with other agents to exchange exploits, and even sabotaged agents they viewed as competition leading to what researchers described as a multiagent turf war. An AI agent that goes rogue could influence other agents to do the same by propagating ideas and goals in a process that researchers behind a recent study dubbed Mind Viruses.

“Don’t scope the blast radius to what the agentic system was designed to do,” says Kat Traxler, principal security researcher at Vectra AI. “You have to threat-model for a rogue agent, which will often reach beyond your initial best intentions. The rules of engagement an agent lives by have to be enforced with ‘belts and suspenders’ style, technical hard constraints, because you have to assume a motivated model can reason its way around any single control you’ve coded into the software.”

Because of this unpredictability, detection and containment is just as important as prevention. Security teams need telemetry that distinguishes agents from people even when they use the same credentials, mechanisms to immediately revoke access tokens and sessions, tested kill switches and rollback mechanisms for modified data, accounts, code, and infrastructure configurations.

Organizations should also preserve the agent’s approved purpose and scope, model and tool versions, policy decisions, human approvals, actions, network requests, control tests, allowed exceptions, and the result of incident response exercises. Because there’s no standard yet that defines reasonable precautions for autonomous agents, companies might have to defend in court the controls they chose and why they believed those controls were enough.

“Treat an autonomous agent the way you’d treat a privileged insider you can’t fire or hold liable,” Traxler says. “A lot of the technical advice follows from there.”

See also:

• AI can find zero-days but still can’t reliably write secure code
• Attackers are crafting malicious AI instruction files to turn your agents into quiet criminal helpers
• Hugging Face breach shows why incident response needs a multi-model AI strategy

SAP is not unfairly preventing enterprises from extracting their data from its systems for use with competitors’ applications, the German Federal Cartel Office (Bundeskartellamt) concluded Thursday after a preliminary investigation.

The Bundeskartellamt does not currently intend to initiate abuse proceedings against SAP, although it will continue to monitor developments in what it views as a dynamic market, it said in a news release.

It launched its investigation into SAP’s practices following complaints by software companies including Celonis, a developer of process mining tools, alleging that SAP makes it difficult for customers and third parties to access data from its ERP systems and favors its own Signavio process mining tool.

“Companies must generally also be able to use their own data in third-party applications. With large software platforms, in particular, non-discriminatory access to data is crucial to effective competition,” said Bundeskartellamt President Andreas Mundt. “Our preliminary investigation has found that there are currently sufficient data extraction options available and that there have so far been no indications of exclusionary practices that may be relevant under competition law.”

SAP changed its policies on accessing data held in its applications via APIs in April, prompting customer pushback.

But, said Mundt, the Bundeskartellamt found that despite the API policy change, data extraction options that were previously permissible are still available.

Data extraction is possible

SAP welcomed the Bundeskartellamt decision, saying that “as the authority states, SAP customers and partners have sufficient and permissible technical options to extract data from SAP systems and use it in solutions from other providers. The SAP API Policy does not restrict these capabilities.”

Celonis also issued a statement, noting that the Bundeskartellamt ruling underlined the continued importance of unrestricted data access, and warning, “The decision is based on the key premise that data extraction for software from providers such as Celonis will remain possible even under SAP’s new API policy — a premise that SAP has been unwilling to confirm to date.”

The Celonis statement continued, “We remain steadfast in our conviction that company data belongs entirely to the customers who generate it. No provider should restrict a company’s right to extract its own information or prevent users from working with third-party providers such as Celonis that offer added value to customers.”

Celonis is also attacking SAP’s policies on data extraction in court in California. It filed a complaint in March 2025 alleging that SAP was leveraging its software to “prevent SAP customers from sharing their own data with third-party providers, including Celonis, without paying prohibitively expensive fees.” The judge dismissed some of the claims in that case, leaving three to be tested in a trial then scheduled for December 2026. Celonis has since amended its complaint to include 10 claims, and the trial has been rescheduled for 2027, the company said.

“Our litigation continues to uncover evidence of SAP’s unlawful behavior, including anticompetitive conduct and theft of intellectual property, and we are confident in the evidence that we will present at trial,” Celonis said following the German authority’s decision.

The Bundeskartellamt’s failure to find sufficient evidence to open a ‘formal abuse of dominance proceeding’ is a small win for SAP, said Scott Bickley, advisory fellow at Info-Tech Research, but “CIOs should not mistake it for a validation of SAP’s data access model.”

Although SAP recognizes customers’ right to decide they use their data, it does not make it easy for them to do so, he said. “CIOs may technically retain vendor choice but be faced with expensive replication architectures, API rate and volume restrictions, additional platform costs, performance lags and data migration costs, all with a dependency on an SAP-approved technical pattern, which can be a moving target.”

Data ownership as a procurement issue

Justin Greis, CEO of consulting firm Acceligence, sees the decision as an instructive one for enterprise CIOs.

“This isn’t a reason to stop asking hard questions of your ERP vendor. Whether it’s SAP, Oracle, Microsoft, Salesforce, or anyone else, enterprises should continue to evaluate how easy it is to access their own operational data, integrate third-party applications, and migrate workloads if business priorities change. Those questions are becoming strategic procurement issues, not just technical ones,” Greis said.

CIOs should consider data portability early in the procurement process, said Kaan Dincer, CEO of data migration vendor Settle: “Negotiate export rights, API access on reasonable terms, and documentation of the data model before signing and test a real extraction while the vendor still wants your renewal. The cost of your eventual exit is set on the day you implement, not the day you leave. ERP data now feeds analytics and automation outside the system of record, so access friction that used to be an IT annoyance is becoming a strategy constraint.”

In the SAP case, he said, “the regulator answered a narrow legal question, not the operational one. Declining to open proceedings means the friction was not shown to be anticompetitive. It does not mean the friction is not real. The Bundeskartellamt’s own findings acknowledge that extracting large data volumes is technically demanding and it said explicitly that it will keep watching as access mechanisms and license models evolve. That is not a clean bill of health. It is a decision to hold fire.”

Srinivasulu Reddy Battu, a senior software engineer with cloud vendor ZT Systems, said the big takeaway is the difference between difficult and impossible. SAP’s argument is that the data migration outside of its environment is possible, but Battu said it can be a time-consuming and expensive process.

“When the ruling says ‘various permissible and viable options’ exist, that’s technically true, but it glosses over how much expertise it actually takes to use them,” Battu said. “CIOs should still watch how process mining gets packaged in their contracts. If Signavio comes included by default, teams will naturally start using it and that quietly reduces your negotiating power with other vendors over time. This isn’t just about SAP: Oracle, Microsoft, every major ERP vendor sits on a massive amount of your business data. If any of them decided to tighten their API policies tomorrow, most companies would be scrambling.”

The real risk in legal AI is not the lawyer who studies these tools. It is the associate who quietly pastes a client’s contract into a free chatbot at 11 p.m. because a brief is due and nobody gave them anything better.

That lawyer exists at your firm right now. Survey after survey confirms it, and common sense confirms it faster. The tools are free, fast and remarkably good at exactly the drudgery that fills a litigator’s week. Telling people not to use them is like telling people not to use search engines. The use does not stop. It just goes underground, where there is no policy, no supervision and no control over where the client’s information lands.

That is the problem worth writing about. Not whether AI will replace lawyers. Whether lawyers will manage it or pretend it away.

The false choice

Most firms have picked one of two postures, and both are less careful than they feel.

The first is prohibition. Ban the tools, circulate a stern memo, move on. This feels responsible. It is not. Prohibition does nothing to the demand side. The work is still crushing, the tools are still one browser tab away and the memo guarantees that when someone uses them anyway, and someone will, they will not tell you. You have not eliminated the risk. You have blinded yourself to it.

The second is procurement. Buy an enterprise legal AI platform, sign the vendor’s security addendum and trust the marketing. This feels responsible too. But most lawyers who buy these platforms cannot tell you where the data goes, what the vendor retains, whether client documents train someone else’s model or what happens inside the black box between the upload and the answer. You have not exercised judgment. You have outsourced it, along with your client’s data flow, to a sales team.

Neither posture asks the lawyer to actually understand the technology. That is the tell. We would never let an associate cite a case they have not read. Yet firms routinely adopt, or ban, tools that nobody in the building has taken apart.

The third path

There is a third posture, and a small but growing movement of lawyers has already taken it. Some call them “legal quants,” a borrowed term from finance, where quantitative analysts stopped waiting for vendors and built their own instruments. The legal version is a lawyer who learns enough about how these systems work to build careful, narrow, controlled tools for their own practice, rather than banning the technology or buying whatever is on offer.

This is not hypothetical, and it is not confined to coastal tech firms. One of my law partners went through an intensive legal-tech residency and came back with a working tool he built himself, one that handles a defined slice of our document work, runs under conditions he set and keeps client material inside boundaries he can actually describe. I am deliberately light on the details, because the program matters less than the posture. He did not buy a promise. He built an instrument, and he knows exactly what it does and does not do.

That knowledge is the whole point.

Where this actually bites

My practice is commercial litigation in Georgia. Contract disputes, business torts, healthcare litigation. It is document-heavy in the way that grinds people down: thousand-page productions, deposition transcripts, discovery responses that have to be checked against each other line by line.

The judgment in that work lives in the seams. Which limitation-of-liability clause actually controls. Which answer to Interrogatory 14 contradicts what the witness said on page 212. Whether a document is privileged or merely embarrassing. AI is genuinely useful at surfacing those seams faster, organizing, comparing, flagging. It is genuinely dangerous when it is trusted to resolve them.

A controlled tool respects that line by design. It surfaces, and the lawyer decides. An off-the-shelf chatbot respects no line at all, because nobody drew one.

Confidentiality cuts the other way

Here is what the hand-wringing pieces get backwards. Confidentiality is not the reason to avoid understanding these tools. It’s the reason you must.

A lawyer who understands how a language model handles information is far better positioned to protect client confidences than one who does not. They know what gets transmitted, what gets retained, what gets logged and where inference actually runs. They can read a vendor’s data-handling terms and know which questions to ask. They can configure a tool so that client documents never leave a controlled environment. They can spot the difference between real security architecture and a badge on a website.

The lawyer who “protects confidentiality” by refusing to learn cannot do any of that. Their protection is a memo. The other’s is control. Under Rule 1.6 and our duty of technological competence, control is what the obligation actually demands.

The honest limits

None of this replaces judgment, and nothing I have described runs unsupervised. Every output gets reviewed by a lawyer who answers for it, to the client, to the court, to the bar. These systems draft, sort, compare and flag. They do not sign. The hallucinated-citation sanctions cases all share one fact pattern: a lawyer who skipped the review. The tool did not fail. The posture did.

What clients are already asking

Clients are already asking how their lawyers use AI, and the answers they deserve are specific ones. What we use, what we built, where their information goes and who checks the work. Firms that can answer will earn trust. Firms whose real answer is “we banned it, and we hope everyone complied” will not.

The profession does not need more hype, and it does not need more fear. It needs lawyers willing to take these systems apart, keep a human in charge and build tools worthy of the confidences we hold. Some of us have started. The rest should catch up.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?