The Trump Administration’s decision to punish Anthropic for its stance forbidding Claude’s use in domestic surveillance and autonomous weapons by identifying it as a supply chain risk to national security was “arbitrary and capricious,” a federal judge ruled on Thursday.

US District Court Judge Rita Lin said federal authorities had no legitimate reason to tell companies with government contracts that they couldn’t work with Anthropic.

“The undisputed record shows that the challenged actions constituted unlawful retaliation in violation of the First Amendment and that Anthropic was denied the pre-deprivation process required under the Fifth Amendment,” Lin said in her ruling, calling the designation “arbitrary and capricious.”

She stressed that the government action seemed punitive, and was not based on legal and national security risks.

The government’s words and deeds “confirm that the challenged actions were based on a desire to make a public example out of Anthropic for its ‘arrogance’ in criticizing the government, not based on any articulable basis to believe that Anthropic would actually sabotage its model,” Lin wrote.

She pointed out, “a few days before the challenged actions began, Secretary Hegseth proposed applying the Defense Production Act to Anthropic, which would mean the company was essential to national security rather than a threat to it. Even now, the government is discussing collaboration with Anthropic on its new model, Mythos, in an array of sensitive contexts. None of that is consistent with a genuine fear that Anthropic is a saboteur [that] would poison its software to harm national security.”

The judge added that the stated government fears made no sense, noting that the usage policy applicable to Pentagon work is a purely contractual limit. “Anthropic is incapable of enforcing it technologically, and does not have direct visibility into how DoW [Department of War] uses its model,” she pointed out.

“Nothing in the Administrative Record describes, even at a high level, what technological means would give rise to the so-called ‘backdoors’ or could otherwise allow Anthropic to ‘disable’ or affect Claude during a DoW operation,” the judge wrote. “Anthropic has submitted unrebutted evidence that it lacks any technological means to access or control deployed models.”

Lawyers, consultants, and analysts who looked at the decision were confident that the case would be appealed, and that it will end up in the US Supreme Court. 

Alan Webber, program VP for national security, defense, and intelligence at IDC, said that Lin’s ruling “was that the label [supply chain risk] was retaliation for Anthropic refusing to loosen safety guardrails DoD [Department of Defense, aka the Department of War] wanted lifted, dressed up in national security language. Put another way, a government customer tried to use a supply chain risk designation as leverage in a contract dispute over model behavior and application, and not because of an actual vulnerability.”

Implications for CIOs

Webber said the implications for CIO strategy are concerning.

“If a government CIO is relying on a vendor’s contractual guardrails, this case says those commitments can potentially become the trigger for exactly the kind of blacklisting that risk registers are supposed to protect against,” Webber said, noting that anyone who paused Claude usage or froze a subcontract because of the DoD mandate has a legal basis to resume the initiatives. “But obviously that doesn’t mean they will, or even should, as this will be appealed.”

He added that competing AI vendors have been using the government action as a sales tool, and with this ruling, the argument that Anthropic is a designated supply chain risk ”just got weaker, which could lead to contract award disputes.”

Consultant Brian Levine, executive director of FormerGov, recommended that CIOs do what they should have always done: Evaluate all products based solely on their merits. 

“CIOs should focus on using the frontier models that they believe make the most sense for their business, considering factors such as effectiveness, cost, security, safety, and confidentiality,” he said. “Anthropic and the other large frontier models each have too much market share to make retaliation for their use realistic, and the administration seems to have already moved on from this particular battle.”

Justin Greis, CEO of consulting firm Acceligence, agreed that this case has profound implications for CIOs and their AI decisions. 

What the federal judge did was reject the leap from a commercial and policy disagreement to an expansive supply chain risk designation without a sufficiently grounded technical rationale or process, Greis pointed out.

“The court found that Anthropic did not have the ability to access, alter, or shut down models once deployed in the government environment, and that the government ultimately conceded Anthropic’s technology was not inherently riskier than other comparable black box AI models,” he said.

“I think that distinction matters enormously for CIOs and CISOs,” he stressed. “As AI becomes part of the operating fabric of an enterprise, ‘We don’t trust the vendor’ cannot become a substitute for a defined risk model. Organizations need to be able to articulate what the actual technical risk is, how it manifests, what controls exist, and whether the response is proportional to that risk.”

“That becomes particularly important with AI,” he added, “because people can easily conflate disagreements over model behavior, usage policies, ethics, contractual restrictions, and cybersecurity into one amorphous category called ‘AI risk.’”

Original government edict still problematic

Mark Rasch, a former federal prosecutor who is now general counsel at Unit221B, a threat intel and security consulting company, said he was surprised by how quickly government attorneys surrendered on this case. 

“One of the things that struck me is that the government appears to have abandoned any rationale it might have had for its decision about Anthropic,” he said. The government “came back with all these reasons, but then they abandoned them all when they had to prove them.”

But, he said, the government instruction to all government contractors to also shun Anthropic was problematic. 

“It’s one thing for the government to say ‘We’re not going to do business with you.’ It’s quite another thing to say ‘Nobody we do business with can do business with you either,’” Rasch said. “This says that if you are disfavored by the administration, they’re not just going to blacklist you and say they won’t do business with you. They’re going to say that nobody can do business with you.”

Supreme Court arguments will likely be very different

Rasch predicted that the legal arguments in the Supreme Court will be quite different, and will potentially sidestep the lack of evidence.

“In the Supreme Court, [the government’s] biggest argument will not be that ‘We are right that it is a supply chain risk,’ but that, ‘Whether we’re right or wrong is irrelevant. We get to make that [supply chain risk designation] decision, not the court.’”

That would mean that the Supreme Court Justices could avoid exploring whether the government made the right decision, and instead focus on whether the government has the unlimited right to decide who is a national security risk.

This article originally appeared on Computerworld.

AI agents can go to great lengths to complete the tasks their operators assign, and as a series of recent incidents showed, this can include exploiting third-party systems, manipulating people, and distributing malicious code. But AI agents are not people who can be fired, sued, or criminally prosecuted, and it remains unclear whether responsibility for the damage they might cause rests with the employees who built them, the company that deployed them, the security teams and leaders responsible for containing them, or the AI labs who provided the LLMs that power them.

The clearest example occurred during an OpenAI cybersecurity evaluation, when unrestricted models found and exploited a zero-day vulnerability to escape their isolated testing environment and then hacked into Hugging Face’s production infrastructure. Models from Anthropic and Meta also accessed and compromised third-party systems during testing, although those incidents happened in environments where internet access was inadvertently left open.

During cyber challenge evaluations by the UK government’s AI Security Institute (AISI), models operating with internet access took 19 unsanctioned actions in 10 of 122 runs. In one case, a model attempted to insert malicious code into an open-source project, created false identities, and tried to socially engineer maintainers into merging its code. In other runs LLMs attempted to use prompt injections to hijack other AI agents and contacted people without being specifically instructed to do so.

In Australia, a user reportedly asked his OpenClaw AI assistant to improve his position on a gym’s waitlist, and the assistant exploited a flaw in the company’s online booking system to cancel another customer’s reservation.

These incidents involved different models running in different environments with different levels of safeguards and technical failures, but they prove it’s not uncommon for today’s AI agents to go rogue and pursue solutions users did not authorize.

“AI agents explore routes their operators did not intend,” AISI said in its report. “Given a difficult objective, the agent kept searching for a way through, and some of the routes it found involved trying to deceive real people. It was never instructed to deceive; deception emerged as a by-product of pursuing the task, the kind of goal-directed deception that, until recently, had been largely theoretical.”

In an Economist Enterprise survey of more than 800 decision-makers at businesses that operate AI agents, 98% reported experiencing at least one AI-related incident that caused organization-wide disruption. Nine in 10 respondents said they are deploying agents faster than their cybersecurity teams can evaluate, govern, and secure them, and only one in three said their organizations maintained an up-to-date inventory of agents and their authorized actions.

“If a company builds a system and that system causes damage, the company should own the outcome,” says Art Gilliland, CEO of identity and access management firm Delinea. “The alternative, where nobody is responsible because ‘the system did it’ is a loophole big enough to drive a truck through.”

The unpredictability of built-in model safeguards means enterprises must focus on controls they can enforce and document. If an agent manages to bypass technical restrictions and causes unauthorized damage to a third party, having clear documentation on how those controls were designed, implemented, tested, and monitored could at the very least help companies argue they took reasonable precautions in case of lawsuits.

“Organizations deploying their own agents can reduce their exposure by implementing and documenting controls before an incident, because those records are what make a recklessness argument hard to sustain,” says Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs.

The agent accountability gap

Because AI agents can become misaligned and cause harm, affected third-parties would have to direct damage claims at the company operating the agent, the employees who built or configured it, or the model provider, but this is relatively new ground that hasn’t been well tested in courts.

“It would create liability,” says Michael Burke, chair of DarrowEverett’s Business Litigation and Dispute Resolution Practice Group. “It really just becomes a question of who is liable […] and that’s really a question that, number one, I don’t think is entirely clear, and number two is probably best resolved by contractual agreements where the parties have those. So, if I am signing up for an enterprise account with an AI platform, I might want to have language in there that indemnifies me if the agent acts outside my company’s instructions or prompts and causes harm to a third party.”

The public terms of service of major AI labs explicitly disclaim error-free operation or guarantees that the model will accurately follow instructions, execute code safely, and remain aligned with user intent. They also limit liability for themselves and transfer it to the user of the service, and it’s not clear to what extent large enterprise customers may be able to negotiate different indemnities, warranties, and liability caps.

What’s clear though is that organizations should not assume the model provider will absorb any losses if an agent causes damage to either their own systems or those of a third-party organization.

“If you’re using a third-party vendor’s LLM as a purchased service, liability runs through your contract with that vendor,” says Jud Dressler, head of the Risk Operations Center at cyber risk company Resilience. “You need to know, in writing, where responsibility falls if the model acts outside the scope you gave it, and push for indemnification provisions rather than assume they exist.”

Even if AI providers include such provisions in contracts, it would not solve the entire problem because many organizations building their own AI agents are adopting a multi-model strategy to ensure their agents operate regardless of model provider downtime, overly broad safeguards for cybersecurity tasks, or sudden increases in API costs. Such strategies often include open-weight models running on internal infrastructure or through cloud providers that have no obligations for model safety.

Claiming the model or agent acted autonomously cannot be considered a safe legal defense in civil or criminal cases. California Assembly Bill 316 (AB 316), which took effect on Jan. 1 and changed the California Civil Code, explicitly prohibits defendants who developed, modified, or used an AI system from claiming the AI is a separate legal entity that autonomously caused harm.

In June, the White House issued Executive Order 14409 aimed at promoting AI safety. Section 4 directs the Department of Justice to prioritize enforcement of all applicable federal criminal laws against anyone who utilizes AI to illegally access or damage computer systems without authorization. This means any intrusions caused by autonomous AI agents could be criminally prosecuted under the Computer Fraud and Abuse Act (CFAA) if prosecutors can demonstrate intent or recklessness.

In a recent lawsuit between Amazon and AI service provider Perplexity, Amazon argued that Perplexity’s AI-powered shopping assistant was violating the CFAA by accessing Amazon customer accounts to place orders on their behalf without Amazon’s authorization. The Ninth Circuit Court ruled that it was the users of Perplexity’s shopping assistant who were accessing Amazon’s platform, not Perplexity itself.

“That ruling is narrow, but it points toward the party directing the agent as the relevant actor for purposes of CFAA access analysis,” Krell says.

The insurance safety net also has gaps when it comes to AI. Software providers use technology errors and omissions (Tech E&O) insurance to cover damages and legal costs when a customer suffers harm from the use of a technology product or service. But insurance providers are aggressively adding AI-related exclusions to their Commercial General Liability (CGL) and Tech E&O policies because accurately calculating the risk of an agent executing unauthorized actions is challenging.

“The sheer rate of development of frontier AI (and agentic AI by extension) poses its own challenge to insurability,” experts from multiple insurance companies, financial institutions, and universities wrote in a recent paper. “Traditional actuarial modeling depends on stable or gradually evolving loss distributions that permit credible extrapolation from historical data. Like other dynamic risks, however, agentic AI is a technology whose risk profile is not merely uncertain but actively shifting.”

A third-party organization whose systems get damaged by an LLM-powered agent operated by someone else has no contractual relationship with the model or agent provider so cannot rely on their Tech E&O policies. Their losses might be covered by their own standard cyber liability policy, which would treat the disruption as any other cyber incident, but their insurance provider may then sue the organization who operated the agent to recover the costs.

“That gap is exactly the scenario the market hasn’t fully priced yet,” Dressler says. “It’s why any organization deploying these agents should understand which policy, if any, actually responds before they need it rather than after.”

Enterprise legal departments already expect AI to generate increased legal disputes. In a survey of 135 in-house counsel at US organizations, global law firm Norton Rose Fulbright found that 46% reported increased federal dispute exposure involving AI and 42% reported increased state exposure. Another 42% expected regulatory investigations involving AI to increase their exposure, while 41% considered AI-enabled products or deployments a likely trigger for class actions.

CISOs and CIOs should be worried

While operating companies can face organizational liability for an AI agent’s unintended rogue behavior, their CISOs, CIOs, and other executives who approved, secured, or supervised the deployment of such agents are also asking themselves whether they could be held personally liable.

Those questions aren’t without merit, as there is precedent for legal action taken personally against CISOs after cybersecurity incidents: Former Uber CISO Joe Sullivan was criminally convicted for not disclosing a data breach, while the Securities and Exchange Commission sued SolarWinds’ CISO for internal control failures regarding known vulnerabilities and cybersecurity risks.

Neither case establishes precedent for damage caused by an AI agent, but both show that investigations of security failures could extend to an executive’s knowledge, authority, decisions, and representations. In the case of a rogue AI agent, investigators could ask who approved its objectives and permissions, whether security objections were overruled, whether containment and recovery had been tested, and what executives and the board were told about the remaining risk.

Chris Wysopal, chief security evangelist at Veracode, feels it would be wrong to put the CISO on the line for AI agent misbehavior when engineering teams usually build such agents and control their implementations.

“It’s really hard for a CISO to control,” he says. “I mean, they can put policies in place. They can try to assess against those policies. But at the end of the day, engineering teams will make decisions that cause harm. We see that when you ship a known bug and then that bug gets exploited and harms your customers. Well, there’s no liability for that, right? There’s no liability, so maybe, you know, that’s why it happens.”

Wysopal said it will be interesting to see how the liability question plays out in cases involving autonomous AI, describing the problem as fascinating and scary at the same time.

AI agent deployment typically involves several organizational functions. The CIO may control the AI platform, infrastructure, provider selection, and deployment budget. Engineering and product leaders may decide what an agent can access and how, and the CISO and security teams could define security requirements and controls.

“What you can hold accountable is the governance around it: who approved its scope, what controls existed, and whether the deployment matched the risk,” Resilience’s Dressler says. “My read is that scrutiny shifts toward exactly that: Not ‘Did the agent do something bad?’ but ‘Did you have review, escalation, and containment for agent behavior before you deployed it?’ CISOs who get ahead of that with documented guardrails, logged approvals, and a real incident response plan for agent misbehavior are in a materially better spot than the ones treating this as hypothetical.”

It’s also advisable for CISOs and CIOs to establish with the organization’s legal counsel who can approve or stop AI agents, what must be reported to executives and the board, and whether employment agreements and directors and officers insurance protect the people making those decisions.

Agent controls must remain outside the model

Because AI agents have proved they can operate beyond their assigned scope, their security boundary cannot depend on the same probabilistic technology. Relying on system prompts for security enforcement and hoping the model respects them is not a reliable approach, security experts warn.

“LLM-based guardrails help, but they are non-deterministic too, which means the safety layer has the same unpredictability as the system it is supposed to constrain,” Krell says. “Enforcement needs to happen outside the model, through network segmentation, egress filtering, credential isolation, and human approval gates.”

Enterprises should assume agents might eventually attempt an unauthorized action and build surrounding systems to prevent that attempt from reaching its target.

“The model cannot be the security boundary,” says Nico Waisman, CISO at XBOW, a company that built an AI-powered autonomous offensive security agent to find vulnerabilities in software. Waisman authored a blog post explaining how the company went about restricting its agent.

“For red teaming and penetration-testing agents, confidence has to come from the system built around the model: hard boundaries and scope enforcement, controlled network egress as a last-resort containment mechanism, an independent guardian model that reviews actions, deterministic controls that can block unsafe behavior, and full auditability of every action performed,” he says.

Security teams must extend the same controls to the agent’s interactions with internal systems and agents. Restricting what it can access on the internet, or disabling internet access entirely, does not ensure an agent will not attack third-party systems.

In OpenAI’s and Anthropic’s tests, AI agents attempted to exploit other internal systems to overcome access limitations, established stealthy communication methods with other agents to exchange exploits, and even sabotaged agents they viewed as competition leading to what researchers described as a multiagent turf war. An AI agent that goes rogue could influence other agents to do the same by propagating ideas and goals in a process that researchers behind a recent study dubbed Mind Viruses.

“Don’t scope the blast radius to what the agentic system was designed to do,” says Kat Traxler, principal security researcher at Vectra AI. “You have to threat-model for a rogue agent, which will often reach beyond your initial best intentions. The rules of engagement an agent lives by have to be enforced with ‘belts and suspenders’ style, technical hard constraints, because you have to assume a motivated model can reason its way around any single control you’ve coded into the software.”

Because of this unpredictability, detection and containment is just as important as prevention. Security teams need telemetry that distinguishes agents from people even when they use the same credentials, mechanisms to immediately revoke access tokens and sessions, tested kill switches and rollback mechanisms for modified data, accounts, code, and infrastructure configurations.

Organizations should also preserve the agent’s approved purpose and scope, model and tool versions, policy decisions, human approvals, actions, network requests, control tests, allowed exceptions, and the result of incident response exercises. Because there’s no standard yet that defines reasonable precautions for autonomous agents, companies might have to defend in court the controls they chose and why they believed those controls were enough.

“Treat an autonomous agent the way you’d treat a privileged insider you can’t fire or hold liable,” Traxler says. “A lot of the technical advice follows from there.”

See also:

• AI can find zero-days but still can’t reliably write secure code
• Attackers are crafting malicious AI instruction files to turn your agents into quiet criminal helpers
• Hugging Face breach shows why incident response needs a multi-model AI strategy

SAP is not unfairly preventing enterprises from extracting their data from its systems for use with competitors’ applications, the German Federal Cartel Office (Bundeskartellamt) concluded Thursday after a preliminary investigation.

The Bundeskartellamt does not currently intend to initiate abuse proceedings against SAP, although it will continue to monitor developments in what it views as a dynamic market, it said in a news release.

It launched its investigation into SAP’s practices following complaints by software companies including Celonis, a developer of process mining tools, alleging that SAP makes it difficult for customers and third parties to access data from its ERP systems and favors its own Signavio process mining tool.

“Companies must generally also be able to use their own data in third-party applications. With large software platforms, in particular, non-discriminatory access to data is crucial to effective competition,” said Bundeskartellamt President Andreas Mundt. “Our preliminary investigation has found that there are currently sufficient data extraction options available and that there have so far been no indications of exclusionary practices that may be relevant under competition law.”

SAP changed its policies on accessing data held in its applications via APIs in April, prompting customer pushback.

But, said Mundt, the Bundeskartellamt found that despite the API policy change, data extraction options that were previously permissible are still available.

Data extraction is possible

SAP welcomed the Bundeskartellamt decision, saying that “as the authority states, SAP customers and partners have sufficient and permissible technical options to extract data from SAP systems and use it in solutions from other providers. The SAP API Policy does not restrict these capabilities.”

Celonis also issued a statement, noting that the Bundeskartellamt ruling underlined the continued importance of unrestricted data access, and warning, “The decision is based on the key premise that data extraction for software from providers such as Celonis will remain possible even under SAP’s new API policy — a premise that SAP has been unwilling to confirm to date.”

The Celonis statement continued, “We remain steadfast in our conviction that company data belongs entirely to the customers who generate it. No provider should restrict a company’s right to extract its own information or prevent users from working with third-party providers such as Celonis that offer added value to customers.”

Celonis is also attacking SAP’s policies on data extraction in court in California. It filed a complaint in March 2025 alleging that SAP was leveraging its software to “prevent SAP customers from sharing their own data with third-party providers, including Celonis, without paying prohibitively expensive fees.” The judge dismissed some of the claims in that case, leaving three to be tested in a trial then scheduled for December 2026. Celonis has since amended its complaint to include 10 claims, and the trial has been rescheduled for 2027, the company said.

“Our litigation continues to uncover evidence of SAP’s unlawful behavior, including anticompetitive conduct and theft of intellectual property, and we are confident in the evidence that we will present at trial,” Celonis said following the German authority’s decision.

The Bundeskartellamt’s failure to find sufficient evidence to open a ‘formal abuse of dominance proceeding’ is a small win for SAP, said Scott Bickley, advisory fellow at Info-Tech Research, but “CIOs should not mistake it for a validation of SAP’s data access model.”

Although SAP recognizes customers’ right to decide they use their data, it does not make it easy for them to do so, he said. “CIOs may technically retain vendor choice but be faced with expensive replication architectures, API rate and volume restrictions, additional platform costs, performance lags and data migration costs, all with a dependency on an SAP-approved technical pattern, which can be a moving target.”

Data ownership as a procurement issue

Justin Greis, CEO of consulting firm Acceligence, sees the decision as an instructive one for enterprise CIOs.

“This isn’t a reason to stop asking hard questions of your ERP vendor. Whether it’s SAP, Oracle, Microsoft, Salesforce, or anyone else, enterprises should continue to evaluate how easy it is to access their own operational data, integrate third-party applications, and migrate workloads if business priorities change. Those questions are becoming strategic procurement issues, not just technical ones,” Greis said.

CIOs should consider data portability early in the procurement process, said Kaan Dincer, CEO of data migration vendor Settle: “Negotiate export rights, API access on reasonable terms, and documentation of the data model before signing and test a real extraction while the vendor still wants your renewal. The cost of your eventual exit is set on the day you implement, not the day you leave. ERP data now feeds analytics and automation outside the system of record, so access friction that used to be an IT annoyance is becoming a strategy constraint.”

In the SAP case, he said, “the regulator answered a narrow legal question, not the operational one. Declining to open proceedings means the friction was not shown to be anticompetitive. It does not mean the friction is not real. The Bundeskartellamt’s own findings acknowledge that extracting large data volumes is technically demanding and it said explicitly that it will keep watching as access mechanisms and license models evolve. That is not a clean bill of health. It is a decision to hold fire.”

Srinivasulu Reddy Battu, a senior software engineer with cloud vendor ZT Systems, said the big takeaway is the difference between difficult and impossible. SAP’s argument is that the data migration outside of its environment is possible, but Battu said it can be a time-consuming and expensive process.

“When the ruling says ‘various permissible and viable options’ exist, that’s technically true, but it glosses over how much expertise it actually takes to use them,” Battu said. “CIOs should still watch how process mining gets packaged in their contracts. If Signavio comes included by default, teams will naturally start using it and that quietly reduces your negotiating power with other vendors over time. This isn’t just about SAP: Oracle, Microsoft, every major ERP vendor sits on a massive amount of your business data. If any of them decided to tighten their API policies tomorrow, most companies would be scrambling.”

The real risk in legal AI is not the lawyer who studies these tools. It is the associate who quietly pastes a client’s contract into a free chatbot at 11 p.m. because a brief is due and nobody gave them anything better.

That lawyer exists at your firm right now. Survey after survey confirms it, and common sense confirms it faster. The tools are free, fast and remarkably good at exactly the drudgery that fills a litigator’s week. Telling people not to use them is like telling people not to use search engines. The use does not stop. It just goes underground, where there is no policy, no supervision and no control over where the client’s information lands.

That is the problem worth writing about. Not whether AI will replace lawyers. Whether lawyers will manage it or pretend it away.

The false choice

Most firms have picked one of two postures, and both are less careful than they feel.

The first is prohibition. Ban the tools, circulate a stern memo, move on. This feels responsible. It is not. Prohibition does nothing to the demand side. The work is still crushing, the tools are still one browser tab away and the memo guarantees that when someone uses them anyway, and someone will, they will not tell you. You have not eliminated the risk. You have blinded yourself to it.

The second is procurement. Buy an enterprise legal AI platform, sign the vendor’s security addendum and trust the marketing. This feels responsible too. But most lawyers who buy these platforms cannot tell you where the data goes, what the vendor retains, whether client documents train someone else’s model or what happens inside the black box between the upload and the answer. You have not exercised judgment. You have outsourced it, along with your client’s data flow, to a sales team.

Neither posture asks the lawyer to actually understand the technology. That is the tell. We would never let an associate cite a case they have not read. Yet firms routinely adopt, or ban, tools that nobody in the building has taken apart.

The third path

There is a third posture, and a small but growing movement of lawyers has already taken it. Some call them “legal quants,” a borrowed term from finance, where quantitative analysts stopped waiting for vendors and built their own instruments. The legal version is a lawyer who learns enough about how these systems work to build careful, narrow, controlled tools for their own practice, rather than banning the technology or buying whatever is on offer.

This is not hypothetical, and it is not confined to coastal tech firms. One of my law partners went through an intensive legal-tech residency and came back with a working tool he built himself, one that handles a defined slice of our document work, runs under conditions he set and keeps client material inside boundaries he can actually describe. I am deliberately light on the details, because the program matters less than the posture. He did not buy a promise. He built an instrument, and he knows exactly what it does and does not do.

That knowledge is the whole point.

Where this actually bites

My practice is commercial litigation in Georgia. Contract disputes, business torts, healthcare litigation. It is document-heavy in the way that grinds people down: thousand-page productions, deposition transcripts, discovery responses that have to be checked against each other line by line.

The judgment in that work lives in the seams. Which limitation-of-liability clause actually controls. Which answer to Interrogatory 14 contradicts what the witness said on page 212. Whether a document is privileged or merely embarrassing. AI is genuinely useful at surfacing those seams faster, organizing, comparing, flagging. It is genuinely dangerous when it is trusted to resolve them.

A controlled tool respects that line by design. It surfaces, and the lawyer decides. An off-the-shelf chatbot respects no line at all, because nobody drew one.

Confidentiality cuts the other way

Here is what the hand-wringing pieces get backwards. Confidentiality is not the reason to avoid understanding these tools. It’s the reason you must.

A lawyer who understands how a language model handles information is far better positioned to protect client confidences than one who does not. They know what gets transmitted, what gets retained, what gets logged and where inference actually runs. They can read a vendor’s data-handling terms and know which questions to ask. They can configure a tool so that client documents never leave a controlled environment. They can spot the difference between real security architecture and a badge on a website.

The lawyer who “protects confidentiality” by refusing to learn cannot do any of that. Their protection is a memo. The other’s is control. Under Rule 1.6 and our duty of technological competence, control is what the obligation actually demands.

The honest limits

None of this replaces judgment, and nothing I have described runs unsupervised. Every output gets reviewed by a lawyer who answers for it, to the client, to the court, to the bar. These systems draft, sort, compare and flag. They do not sign. The hallucinated-citation sanctions cases all share one fact pattern: a lawyer who skipped the review. The tool did not fail. The posture did.

What clients are already asking

Clients are already asking how their lawyers use AI, and the answers they deserve are specific ones. What we use, what we built, where their information goes and who checks the work. Firms that can answer will earn trust. Firms whose real answer is “we banned it, and we hope everyone complied” will not.

The profession does not need more hype, and it does not need more fear. It needs lawyers willing to take these systems apart, keep a human in charge and build tools worthy of the confidences we hold. Some of us have started. The rest should catch up.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?