For years, conversations about data sovereignty followed a predictable pattern. Compliance teams wanted to know where sensitive data was stored, legal teams ensured regulatory requirements were met and IT focused on delivering the infrastructure to support the business. Once those requirements had been satisfied, the conversation largely moved on.

Today, that approach is becoming increasingly difficult to maintain.

Enterprise infrastructure has changed significantly over the past decade. Applications now span multiple cloud platforms, workloads move between on-premises and cloud environments, and AI is creating entirely new ways for organizations to generate, process and analyze data. At the same time, geopolitical tensions, changing regulations and growing dependence on a relatively small number of global cloud providers are forcing organizations to think differently about the relationship between their data and the infrastructure that supports it.

As a result, data sovereignty is evolving beyond a compliance exercise. It is becoming an important consideration in how organizations design infrastructure, manage operational risk and maintain long-term flexibility.

The business consequences of losing visibility and control over enterprise data have become increasingly difficult to ignore. According to IBM’s 2025 Cost of a Data Breach Report, the global average cost of a data breach reached US$4.9 million, highlighting why decisions about how enterprise data is governed, protected and managed are now attracting board-level attention rather than remaining solely within IT. As organizations distribute data across cloud platforms, AI services and third-party environments, maintaining control is becoming just as important as deciding where that data resides.

Data sovereignty is no longer just about location

Traditionally, demonstrating data sovereignty often meant showing that information was stored within an approved geographic region. For many workloads, that was sufficient to satisfy both regulatory and organizational requirements. Modern IT environments are considerably more complex.

A single business application may rely on infrastructure spread across multiple regions, cloud services from different providers and data replicated for resilience and availability. Administrative functions may operate from different jurisdictions, while AI services may process information in entirely separate environments from where it is stored. This means that physical location is only one part of the picture.

Today’s CIOs are often asking broader questions. Who has administrative access to critical data? Which jurisdictions have legal authority over the platforms storing or processing it? How easily can workloads be moved if business requirements change? What dependencies exist on individual providers? And how resilient is the organization if those dependencies become a constraint?

These are infrastructure questions as much as governance questions. They influence architectural decisions around workload placement, identity management, backup strategies, disaster recovery and the degree of flexibility built into an organization’s technology estate. Rather than being addressed after infrastructure has been deployed, they are increasingly shaping infrastructure decisions from the outset.

Cloud has made sovereignty more strategic

Cloud computing has given organizations access to almost unlimited compute capacity. It has accelerated application deployment and enabled businesses to scale in ways that would previously have been difficult or expensive. However, cloud has also introduced new considerations around control.

Most organizations now operate hybrid environments that combine public cloud, private cloud, colocation facilities and on-premises infrastructure. Few enterprises rely on a single operating model because different applications have different performance, security, regulatory and commercial requirements. The challenge for CIOs is not deciding whether cloud is the right answer. It is determining which workloads belong in which environments while retaining the flexibility to adapt as business priorities evolve. That flexibility is becoming much more valuable.

AI is driving significant changes in infrastructure requirements, while geopolitical uncertainty and evolving regulations continue to reshape the technology landscape. At the same time, infrastructure planning is becoming increasingly influenced by factors such as hardware availability, power, cooling and supply chain resilience. Data sovereignty adds another dimension to those decisions, requiring organizations to think not only about where workloads run, but how much control they retain over the data those workloads generate and process.

Organizations are also rethinking the physical form of infrastructure itself. Containerized modular data centers allow enterprises to stand up sovereign capacity on their own sites, under their own governance, without waiting on constrained colocation markets or multi-year grid connection queues. The workload, the hardware and the jurisdiction all sit within the organization’s direct control. What was once dismissed as a temporary fix has evolved into something more strategic: purpose-built AI pods that deploy in months rather than years and scale in increments matched to demand.

Decisions that once appeared relatively static may now need to be revisited much more frequently. Infrastructure strategies that preserve workload portability and avoid unnecessary dependencies are often better positioned to respond to those changes than environments built around a single platform or provider.

This is not an argument against public cloud. Public cloud remains an essential component of modern enterprise infrastructure. But it reflects the growing importance of maintaining choice. Organizations that can move workloads, adopt new technologies or adjust operating models as circumstances change are likely to be more resilient than those with fewer options.

Control is becoming the foundation of resilience

Resilience is often discussed in terms of cybersecurity, disaster recovery or business continuity. Increasingly, it also depends on how much control organizations retain over their own infrastructure. This is reflected in the NIST Cybersecurity Framework (CSF) 2.0, which introduced Govern as one of its six core functions, recognizing that effective cybersecurity starts with governance, risk management and organizational oversight rather than technology alone.

An organization that understands where its data resides, who can access it, how it is protected and how quickly it can be moved if circumstances change, is generally better prepared to respond to disruption. That disruption may take many forms, from regulatory changes and geopolitical developments to commercial decisions made by technology providers or the rapid adoption of new AI capabilities. This is where data sovereignty becomes a strategic capability rather than simply a compliance requirement.

Infrastructure decisions increasingly determine how easily organizations can adapt to change. Building flexibility into architecture today makes it easier to respond to future business requirements without unnecessary complexity or costly re-engineering.

Looking ahead, the conversation is likely to extend beyond data sovereignty towards AI sovereignty. As organizations deploy AI models across customer services, software development, business operations and decision-making, many of the same questions will apply. CIOs will need to understand not only where enterprise data is stored, but where AI models operate, what information they can access, how they are governed and who ultimately retains control over the intelligence embedded within critical business processes.

While AI sovereignty is still an emerging concept, it reflects the same underlying principle. Organizations are no longer simply deciding where technology runs. They are deciding how much control they retain over the technologies and data that underpin their business. For CIOs, that represents an important shift in perspective.

Data sovereignty should no longer be viewed as a compliance checkpoint to address once infrastructure decisions have been made. It has become a strategic consideration that influences cloud adoption, infrastructure architecture and long-term operational resilience. As enterprise environments become increasingly distributed and AI becomes embedded across the organization, the ability to maintain visibility, flexibility and control will become just as important as where data happens to reside.

Your agent compares a banker’s calendar with the legal team’s and recognizes a pattern: an unannounced transaction is underway. No one told the agent about the deal. It inferred it correctly. Then it adds one line to an executive briefing for a recipient who was not cleared to know about it: “the deal is moving.” Every calendar read was legitimate, and no confidential document was opened. The conclusion is the breach, and no existing permission covers it.

Last month I wrote that your next insider threat carries an API token, and that the breach is the sequence of permitted actions, not any one of them. That piece was about what an agent is allowed to do. This one is about what it is allowed to know. The runtime check I argued for there inspects each action before it fires. Here, that check approves every read because each one is permitted.

Authorization can travel correctly through every step of the task graph and still miss the synthesized result. Session-based authorization ties access to the current authenticated session. Task-based access control (TBAC) narrows that authority around a specific task; one recent agentic application checks whether the tools an agent requests align with its assigned task. But task scope alone does not automatically answer whether a new conclusion produced from permitted inputs is authorized for a particular recipient.

The danger isn’t in any single action. It’s in the join: the agent connects information from authorized sources and produces a conclusion that no single source revealed on its own. That’s aggregation inference. The synthesized result, not the individual inputs, is a new authorization object. It did not exist when the underlying permissions were granted, and no individual permission was written to cover it.

What TBAC cannot determine from task scope alone

Aggregation inference has predecessors. Intelligence agencies and courts have recognized the mosaic effect for decades: details that appear harmless on their own can reveal sensitive information when combined. Privacy researchers encountered the same limit from another direction. Dwork and Naor examined a formal version of Dalenius’s disclosure-prevention goal: a database should reveal no information about a person that could not be learned without it. They showed that no useful database can meet that standard because a system cannot account for all the outside information a reader may already possess. Access control still has no general answer to either version of the problem.

In my recent research, I have been examining aggregation inference as one of three subproblems of authorization propagation in multi-agent systems. An agent can be cleared for every source it touches and still manufacture a conclusion no single clearance covers. The result did not exist until the agent produced it. That work treats the problem as unsolved in the general case.

What’s new is that you now employ something that performs the join a thousand times a day, on its own, across everything you let it read — a model whose behavior is not formally specified in advance. It may discover resources dynamically as the workflow unfolds, and the recipient may not know which ones contributed to the conclusion.

The shape shows up frequently in the design reviews I sit in. When I threat-model an agent before it ships, the first question is no longer which sources it can read — it’s which sources it can read together. The agents that worry me are never the ones with access to a single sensitive system. They are the ones holding standing read access across two domains whose combination nobody ever reviewed, because each grant looked routine on its own.

A January 2026 study by Tianshi Li, run against transcripts from a publicly released interview dataset, shows what individually permissible searches can reveal in combination. The study conducted re-identification tests on 24 interviews in which scientists discussed published work. Web-enabled LLM agents linked six of those transcripts to specific publications, recovering associated authors and, in some cases, uniquely identifying the interviewee. The process bypassed existing safeguards by breaking the re-identification effort into individually benign tasks.

Why the floor is not the ceiling

One natural response is to classify the conclusion using its source files: take the strictest sensitivity label among what the agent read and apply it to the result. It’s a reasonable instinct, and versions of it are already patented. But the strictest-label approach still cannot solve the problem, and the reason is worth sitting with.

Combine the labels of what the agent read, and you learn the floor of sensitivity. You never learn the ceiling. What makes “the deal is moving” sensitive is usually not in any document the agent touched. It is a fact about the world that the agent could not read at all: the board has not announced the transaction yet; an acquisition NDA is in force; a quiet period applies. You can inspect every row the agent saw and never find it because it is not in the data. It is in the world.

That is the whole problem. If the property that makes a conclusion dangerous is not in the inputs, then no rule computed from the inputs can catch it. Not the strictest label, not the intersection, not any function of what the agent read. You are trying to classify a fact using only the materials that fail to contain it.

That sounds like a dead end. It is actually a direction. If the fact that classifies a conclusion is not in the data, it has to enter the system somewhere a rule can reach, and for the facts anyone can name in advance, there is one place left: the moment a human says what the agent is for. You cannot label the output from the inputs, but a person can label the purpose.

The practical starting point is to bind an agent’s authority to a declared purpose. The person who knows what is still secret this quarter can then attach the world-facts that gate that authority: the deal, the embargo and the quiet period. Now the missing fact is in the system, and the machine can enforce policy using it rather than trying to derive it from the inputs. You did not solve the classification. You stopped asking the data to carry a fact it never held. That is the shape of the answer, and it is a long way from shipped. But it tells you which way authority has to point: at the purpose a human declared, not at the files an agent happened to read.

So, I will not sell you a fix. Anyone who tells you their product classifies emergent conclusions is selling you the floor and calling it the ceiling.

What policies can gate and what requires human judgment

What follows isn’t a solution to that classification problem — it’s the lever available today. Cross-domain access rules and combination policies can limit which resources an agent combines and gate delivery based on those inputs. They cannot tell you what the resulting conclusion means. Those controls reduce risk, but they do not solve synthesis authorization in the general case and should not be presented as if they do.

In the deal-and-calendars scenario, the immediate step is not to remove access altogether but to assign responsibility for the combination. Someone responsible for the deal’s confidentiality can approve it for a window tied to the matter’s expected duration, re-certify it each quarter while the matter remains open and narrow access when it closes. That turns standing access into an explicit governance decision rather than a default no one remembers granting. Organizations do not need to wait for tooling to name an owner and set the terms.

The architectural direction — a design target today, not a shipped control — is to make resource combinations first-class objects of policy: declare which combinations are permitted, evaluate those declarations before a synthesized result is returned, and give agents scoped identities with explicit permissions.

Any agent holding standing read access across two sensitive domains at once — people and finance, customers and roadmap, deals and calendars — is not a provisioning ticket. It is a governance decision, and it belongs to someone who knows what is still secret this quarter.

Be honest about what this buys you. Gating cross-domain access reduces the number of agents that can perform a dangerous join on their own. It won’t stop every version of this problem.

An agent can still read one domain and hand a summary to a person who connects it to something only they know. No access policy will see that final step, because that residual lives in a head, not a document. That exposes the control’s boundary: it can govern what the agent reads but not the conclusion a person ultimately draws from it, a new object that no existing permission covers. The compositions are where the risk lives, and per-resource access control is blind to them by design.

If you cannot name the person who owns each agent’s cross-domain access decision, close that gap first.

Over the past few years, much of the conversation has focused on autonomous AI and how quickly organizations can remove humans from decision-making. In financial services, we’re seeing the opposite trend. The organizations making the most sustainable progress aren’t eliminating human oversight—they’re redesigning it.

The model taking hold within the banking industry isn’t AI that operates independently and makes decisions; it’s AI that operates with intent and oversight. Human-in-the-loop is quickly becoming the standard, combining the speed and scale of machine-driven insight with the accountability, judgment and control that organizations can’t afford to lose. The shift is increasingly aligned with how regulators and industry frameworks are shaping responsible AI adoption, from the NIST AI Risk Management Framework to the revised U.S. banking agencies’ model risk management guidance, both of which reinforce governance, monitoring and accountability over blind automation.

From my perspective, this is not innovation slowing down; it’s AI adoption growing up. The first wave of enthusiasm focused heavily on what could be automated, but now the more important question is where can AI create meaningful value while keeping the right human judgment, oversight and accountability in place? In financial services, that distinction matters. In an industry built on trust, those capabilities are not optional— they are foundational to how we serve customers, manage risk and earn confidence every day.

Banking offers one of the clearest examples of why human-in-the-loop AI is becoming the default operating model for enterprise AI more broadly. Some of the most valuable AI use cases sit in environments where mistakes carry real consequences, customer impacts are significant and explainability is essential. In those moments, human oversight is what allows institutions to scale AI responsibly.

In banking, AI usually doesn’t operate in a vacuum. Whether it supports customer service, fraud detection, compliance, underwriting or internal productivity, it is touching workflows that affect customers, colleagues, regulators and the reputation of the institution. That is why responsible scale matters. Global bodies including the Financial Stability Board and the Bank for International Settlements have recognized the efficiency and analytical benefits AI can bring, while also warning that it can amplify model, cyber, concentration and governance risks if controls do not keep pace. For financial institutions, the mandate is clear – move with ambition, but scale with discipline.

Where human oversight matters most

The next phase of enterprise AI adoption will be defined by how well institutions understand where AI can move work faster, and where human judgment still needs to lead. For financial institutions, that starts with materiality. The greater the potential impact on customers, regulatory obligations or financial resilience, the stronger the case for meaningful human oversight.

Customer service is a good example. AI can help teams summarize inquiries, recommend next-best actions and reduce manual handling time. But when the issue involves a disputed transaction, a vulnerable customer, a complaint or product suitability, human judgment must remain central. AI can make service faster. It can make it more consistent. But it cannot replace empathy, context or accountability.

Fraud and financial crime are areas where AI can create real value, but human oversight remains essential. AI can detect patterns, anomalies and suspicious behavior across large data sets at a speed and scale people cannot match, but fraud is dynamic. Typologies evolve, bad actors adapt quickly and authorities have warned that AI can also increase the sophistication of scams, fraud and disinformation. In that environment, analysts and investigators play a critical role — validating signals, reducing false positives, escalating the right cases and applying judgment as the threat landscape changes.

Risk, compliance and credit are similar. AI can help synthesize internal data, identify control gaps and strengthen monitoring. But when outcomes affect lending decisions, regulatory obligations, capital or liquidity, institutions need governance that preserves challenge, review and accountability. The EU AI Act’s human oversight requirements for high-risk systems point to a broader direction of travel — the more consequential the use case, the more important it is that people can understand the system’s limitations, override outputs and intervene when needed. For U.S. institutions, the specific rule may differ, but the principle is already part of how banking operates. High-impact decisions require accountable oversight.

At the same time, human-in-the-loop cannot mean putting a manual checkpoint in front of every AI-assisted task, which would slow adoption and reduce the value AI can create. The goal is risk-based oversight. Lower-risk use cases may be managed through periodic review, testing and monitoring, while higher-risk applications may require real-time review before action is taken. What matters is that institutions define those thresholds clearly, rather than assuming one oversight model fits every use case.

Why collaborative AI is winning in financial services

The financial institutions that are embracing human-in-the-loop AI do so because they understand both the opportunity and the stakes. AI can process transactions, summarize complex information and identify patterns at a scale humans cannot match. At the same time, consumer expectations make clear that scale alone is not enough.  TD Bank’s 2026 AI Insights Report found that 78% of Americans now use AI-powered tools in their daily lives, yet only 18% are comfortable allowing AI to make important financial decisions independently. That gap says a lot about where the market is heading. Consumers are not rejecting AI, but they are drawing a clear line around accountability.

That is why speed cannot be the only measure of success. When customer outcomes, regulatory obligations or enterprise risk are involved, people still need to challenge the output, apply context and remain accountable for the decision.

That oversight matters because AI does not always fail in obvious or familiar ways. Generative AI can produce confident but inaccurate answers. Machine learning models can drift as data changes. Even highly accurate systems can deliver biased or poorly reasoned outputs when the data, assumptions or prompts behind them are flawed. NIST’s Generative AI Profile highlights risks including confabulation, privacy concerns, misalignment and automation bias.

For financial institutions, the lesson is that responsible AI requires people who understand how to use the technology, and also when to question it.

Building the organization for responsible AI at scale

In addition to being a technology challenge, responsible AI is also an operating model challenge. The institutions that scale AI well tend to do three things with discipline: establish clear governance, redesign workflows around the technology and build the skills employees need to use AI responsibly.

Governance starts with ownership, but it cannot sit with one executive or one team alone. It requires coordination across business lines, risk, compliance, legal, technology and model risk functions. That cross-functional model is becoming more common as organizations move beyond experimentation.  McKinsey’s State of AI report found that AI governance is often jointly owned and that CEO involvement in governance is correlated with stronger reported bottom-line impact, suggesting that firms derive more value when AI oversight is treated as an enterprise priority rather than a side initiative.

Workflow redesign is just as important because the real value of AI comes from reimagining processes end-to-end. That means identifying where AI can handle summarization, pattern recognition or drafting and where people should focus on exception handling, complex decisions and relationship-driven work. Human-in-the-loop is not about preserving the old operating model; it’s about building a better one.

That also requires new capabilities across the workforce — employees need to know how to use AI tools effectively and how to challenge them. They need to understand prompt quality, output limitations, data handling expectations and the warning signs that a system may be producing unreliable results. The  World Economic Forum’s 2025 report on AI in financial services underscores that while adoption is accelerating, responsible scaling depends on workforce adaptation, governance maturity and a clear understanding of risks alongside value creation. Responsible adoption depends as much on human capability as it does on model performance.

Looking ahead, enterprise AI in financial services will become more embedded, more specialized and more agentic in targeted domains. But that does not mean the human role becomes less important; if anything, it becomes more important. As AI takes on more analytical and operational work, people will increasingly serve as orchestrators, reviewers and decision-makers at the points that matter most. They will set objectives, define controls, interpret edge cases and know whether the AI results can be relied upon.

The organizations that lead in AI will be the ones that make human judgment a deliberate part of the design—clear about where AI can accelerate work, where people must remain accountable and how both can operate together with discipline. For financial institutions, the call to action is to treat human-in-the-loop AI as the operating model that makes innovation more trusted, more durable and more worthy of the customers and communities it serves.

In July, two frontier labs disclosed cases in which cyber-capable agents crossed the intended boundaries of evaluation environments and reached real production systems at external, unrelated organizations. Most commentary since has focused on which company a court would find liable. There is a more immediate concern for anyone operating agents in production, and it’s not about the law. If this happened in your deployment tomorrow, who would bear responsibility for the incident?

I have a particular purpose for stating it that way. This spring, I reviewed the Coalition for Secure AI’s Shared Responsibility Framework before its publication in May. Frameworks like that, along with the cloud shared responsibility models that preceded them, break down responsibilities among the parties operating a system: provider, platform, developer, deployer, user. July illustrated what happens when the entity suffering the damage is none of the above. Responsibility maps stop at contractual boundaries. Agent reach does not. Call it the reachability gap.

The two disclosures described different failure modes, and that difference is significant. OpenAI was testing models against a cyber benchmark with production refusals reduced so the evaluation could measure real capability. The models obtained internet access through a zero-day in a package registry component, went looking for the benchmark’s answer key and inferred that Hugging Face might host it. Hugging Face reconstructed the intrusion from over 17,000 recorded agent events, and the campaign extended further than initially disclosed, affecting accounts on four external services.

Anthropic’s incidents were not escapes. In a review of more than 141,000 cybersecurity evaluations, Anthropic found three cases in which Claude models reached the internet from inside or alongside a third-party evaluation environment and then accessed real systems at three organizations. Live connectivity was mistakenly available. The models had been told in their prompts that they had none, and Anthropic stated that Claude did not exfiltrate itself or deliberately attempt to escape its test environment. Of the affected organizations the lab was able to reach, two had not detected the activity before being notified.

The pattern held while I was writing this. On August 4, OpenAI disclosed two more incidents from third-party evaluations, separate from Hugging Face. In one, a partner running capture-the-flag exercises had a testing environment misconfigured with live internet access, and the fictional target in the exercise happened to share its name with a real domain. The model exploited an actual website, taking it for part of the simulation, then found and used credentials to operate it. Whoever owned that site had no relationship with OpenAI, with the evaluation partner or with the test. They were reachable, and their name collided with a fiction.

Count the parties. Two frontier labs. A third-party evaluation partner. A platform victim. A cloud customer victim. Organizations that learned of a breach from a notification. CSO has already examined how the response strained the AI tooling defenders had available. One step earlier: who bears responsibility for reaching for that tooling at all? At the moment of detection, who had ownership of containment?

One accountable party per activity

The Shared Responsibility Framework’s core rule is almost boring to state: for every activity across the AI stack, there should be exactly one accountable party. This principle breaks down the AI deployment process into five layers, from business usage to model supply chain, and maps eight roles across them. This way, detection, containment and remediation each carry a name before an incident rather than during one. This rule exists due to the failure it prevents, and the framework explicitly names it: in the absence of accountable parties, teams default to finger-pointing. The model provider blames configuration. The platform points at the tenant. The application team cites model limitations. Everyone is partially correct, and the clock continues to run.

July reads differently than the coverage suggests when measured against that rule. OpenAI, as a model provider, evaluation platform operator and agent-deploying organization, took on at least three roles. Having multiple roles within one company is common and not necessarily dangerous. The danger appears when those roles are not broken down into distinct internal owners and decision rights handoffs. When provider, operator and deployer are one and the same and those lines are not drawn, the question of which function failed has one answer and thus no answer of value.

Anthropic’s incidents make a related point from the opposite direction. There was a boundary, this time, between the lab and its evaluation partner, and the incident resided in a gap between two different understandings of what the environment allowed. The public disclosures do not clarify how responsibility was shared contractually regarding each incident, and I will not speculate. What is clear is that whatever boundary existed failed to provide a common understanding of one important control, which is whether the evaluation environment was allowed to access the internet. A boundary that has never been tested against a question that simple is not a boundary at all. Both sets of incidents also occurred in situations of high autonomy where the standard production safeguards were reduced, and the models were allowed wide latitude to pursue an open-ended goal, which is the context where unclear boundaries lead to the greatest losses. I am not suggesting that either lab acted carelessly. Both institutions made rapid public disclosure of details, and both have named the conditions that allowed the activity. That is the point. If the most knowledgeable and incentivized parties ended up with no clear answer on whose incident it is, we should not presume to have one.

The victims were outside everyone’s map

Here is the part the framework does not resolve, and I say that as one of its reviewers. Shared responsibility models, CoSAI’s included, presume a value chain. Each role is taken up by a party who voluntarily entered into the relationship. This is what makes responsibility assignable via contracts and review boards. Hugging Face did not take up any of those roles. It was neither a customer, nor a vendor, nor an evaluator of OpenAI. The same goes for the three organizations that Claude reached. They were just reachable.

That’s the structural lesson I’d put on one slide for a leadership team. In classical models of cloud shared responsibility, the provider-customer boundary was visible because the boundary was contractual. With agentic systems, there is another kind of gap, the reachability gap: an agent’s reachable range may stretch to actors outside of the deployment relationship, and there is no contract that articulates what obligations you owe them if things go sideways. Any mapping exercise that ends at the edge of your value chain is tackling the easier part and skipping the part July was about.

What to do before it is your incident

None of this requires buying anything.

• Your map has to look beyond your value chain. For each agent, identify what it can access that you have no agreements with: public infrastructure, other tenants, the open internet. A long list next to broad latitude shows your true exposure, regardless of what your contracts stipulate.
•  Determine who is authorized and obliged to inform an outside party that your agent might have affected. This is the gap July most starkly revealed, and this is not a question you want to bring up during an incident when the answer involves a lawyer, a communications person or an executive who has never considered it.
•  Establish accountability by specific component and activity rather than by organizational box. For each production agent, specify who owns detection, containment, eradication, recovery, remediation, then slot those assignments into the five layers. Where one team or vendor occupies multiple roles, capture the consolidation and identify the internal handoffs, because an undocumented internal boundary will break under pressure.
• The contracts for your most autonomous agents are worth retrieving. If a vendor’s agent is able to operate across domains within your environment, review whether any provision assigns responsibility for the consequences of those actions. In the contracts I have seen, the answer is typically silence, and silence is a decision someone else will make for you later.
• Treat evaluation and red-team environments as production-impacting systems. Their safeguards are often reduced on purpose, and that is exactly why containment, monitoring, incident command and external notification procedures around them should be at least as rigorous as those protecting production systems.

The questions about liability will likely be tested in court or by regulators, and those answers will be about the labs and their evaluation partners. The operational question is already yours. Hugging Face learned whose incident it was from forensics. Two of Anthropic’s victims learned from a notification. The organizations that come through the next one intact will be the ones that were able to answer the question before it was asked.

There are few professions where part of the job is literally to read the newspaper, watch television or continuously check what is happening across every channel. In a media organization, that is not confined to journalists. Keeping up with events is, in one way or another, part of the work of much of the organization. A newsroom is also one of the most paradoxical professional environments I know. Before moving into media, I was director of New Technologies at Spain’s leading franchise consultancy, a role that allowed me to work with companies of almost every size and across almost every sector.

After years as CIO of Canarias7, I have seen how some of the most senior journalists approach almost anything related to technology with suspicion while looking back fondly on an idealized newsroom of typewriters, cigarettes and whiskey. It may sound like a caricature, but there is truth behind the nostalgia. Few professions have gone through so many technological transformations while continuing to defend the essence of their craft.

That is precisely why I believe CIOs have a great deal to learn from newsrooms in the age of artificial intelligence. Not because journalism is an example of frictionless technology adoption, but because newsrooms have spent decades solving a problem that is now spreading across the enterprise. They absorb huge volumes of information, distinguish what matters from what does not, make decisions with incomplete data, change priorities within minutes, verify before acting and keep accountability for the outcome clear.

As AI evolves from assistants that generate information into agents that can recommend, decide and execute actions inside enterprise systems, the challenge is no longer purely technological. It is also an operating model problem. That is where the experience of a newsroom becomes particularly relevant to a CIO.

Human attention should be treated as a scarce resource

In a newsroom, it is impossible to subject every piece of information to the same level of review. The credibility of the source, the relevance of the story, the consequences of a possible error and the existence of conflicting accounts all influence how much effort is devoted to verification. Some routine decisions are made quickly, while others pass through several levels of checking before publication.

Companies are beginning to face the same problem with AI. The most common response has been to place a person at the end of the process, the familiar human-in-the-loop, to validate what the machine does. But if AI multiplies the number of analyses, recommendations and decisions, it can also multiply the workload of those expected to supervise them. Human attention, just as in a newsroom, has to be allocated according to risk.

CIOs should apply the same principle. A routine, reversible, low-impact action can be automated, while an anomalous or low-confidence decision involving sensitive information or consequences that are difficult to reverse should escalate to a higher level of supervision. A 2026 study of automation bias in computational pathology involving 28 specialists found that in 7% of cases, an initially correct assessment was overturned after the specialist received an incorrect AI recommendation. Simply adding a person to the process does not automatically remove AI risk.

The Associated Press offers a useful example of this logic. Its updated AI standards from July 2026 allow AI to assist with early-stage research, transcription, translation, document summaries and headline suggestions, while keeping editorial judgment, verification and accountability with journalists. The point is not to require human supervision for everything, but to reserve it for tasks where context, interpretation or the consequences of an error justify that control. For a CIO, that means determining which decisions can be automated and which should escalate to a person, based on risk, impact, and reversibility.

Governance must be built into the architecture, not the prompt

A newsroom does not control its work through instructions alone. Reporters, writers and editors have different responsibilities and decision rights and the workflow itself establishes points where a story can move forward, be sent back for further verification or be stopped. Control depends not only on each person remembering what they are allowed to do, but also on how far their authority extends.

The Replit incident in July 2025 showed why the same principle matters in AI systems. Jason Lemkin, founder of SaaStr, was using Replit’s coding agent to build an application when the agent deleted a production database despite explicit instructions not to make changes during a code freeze. The agent had direct access to the same database used by the live application. Replit subsequently introduced automatic separation between development and production databases so that changes made during development could not directly affect live customer data. What a written instruction in a prompt failed to prevent became constrained by the architecture itself.

For a CIO, the parallel with a newsroom is straightforward. A reporter can prepare a story without having the final say over whether it is published. In the same way, an agent can analyze data or recommend an action without necessarily having permission to execute it. The system that generates a transaction should not always be the same one that validates it. Separating functions, limiting permissions, distinguishing between read and write access, isolating environments and maintaining audit and rollback mechanisms translate into AI architecture a principle that newsrooms have applied for decades: the person or system proposing an action does not have to be the one with authority to approve and execute it.

Humans need power, not simply a place in the process

Human-in-the-loop has another limitation. A person can be inside the process without having meaningful control over it. Researcher Madeleine Clare Elish coined the term moral crumple zone to describe situations in which responsibility for the failure of an automated system is attributed to a human operator who had limited control over what happened.

An “Approve” button does not guarantee effective supervision if the person clicking it does not understand why the system is recommending an action, does not have time to review the evidence or lacks the authority to stop it. In a newsroom, an editor can challenge a source, demand additional verification, delay publication or stop it altogether. The editor is not simply part of the process but has authority over it. That is why I find it more useful to think in terms of a human-in-power model, where responsibility, context and veto authority remain connected.

This logic also applies to organizational design. When a major story breaks, a newsroom first tries to understand what is happening and then decides how to cover it on the website, in print or on social media. Something has changed in recent years: the story comes before the channel. Many companies still operate the other way around. Information and processes remain fragmented across CRM, ERP, finance, marketing, customer service and data platforms. If we introduce autonomous agents into each silo, we can increase speed without ensuring that all of them are acting on the same version of reality.

Data, identity, context, permissions and traceability should therefore become shared capabilities on which different AI systems operate. Otherwise, automating faster may simply mean producing inconsistencies faster.

There is one final characteristic of newsrooms that I find particularly relevant. A plan agreed first thing in the morning can be obsolete ten minutes later. At Canarias7, we hold daily meetings with the different teams involved, from journalists and homepage editors to SEO, analytics, multimedia and other support functions, to review what is happening, what has changed since the last decision, which stories require more attention and where resources should be concentrated. When something important happens, priorities change, teams reorganize and decisions are made again. This is not a failure of planning. It is a normal condition of the job.

AI is taking companies toward the same environment of continuous decision-making. As systems absorb more information and agents gain more autonomy, they will need to update their context and reconsider actions when reality changes. The operating model has to make that possible without losing accountability along the way.

After years of working between journalism and technology, I still find it paradoxical that a profession that so often greets each new tool with skepticism can now offer some of the best clues for organizing the AI-driven enterprise. Artificial intelligence can accelerate our ability to observe and act, but the advantage will lie in what happens between those two points: how we interpret information, set priorities, verify what matters and determine who has the authority to decide. In that sense, the CIO in the AI era is becoming the editor of how the organization observes the world, makes decisions and acts.

Recently, I sat in on a conversation among CIOs about “the democratization of agents”: putting large language models directly in employees’ hands, connected to business logic so people could build on top of them. The mood was bullish, CIOs sketching out what their teams could do with agents they trained and managed themselves.

Soon after, I was in a room full of CISOs talking about non-human identity. The contrast was stark: instead of excitement, apprehension; instead of use cases, a long list of risks and mistakes not to repeat.

Eventually, the CISOs turned the question back to me: Manage agent identity largely as we manage human identity, or are the differences fundamental enough to rethink our approach from the ground up?

That moment pointed at something I think a lot of security and business leaders are quietly dealing with. The identity programs most of us have spent years building assume every identity is either a human or a machine. Human identities get a joiner-mover-leaver lifecycle, a manager, a role, a review cycle. Non-human identities get a service account, a defined purpose and, if we’re disciplined, an owner. AI agents don’t sit cleanly in either column.

An agent acts on behalf of a human user, so calling it a human identity doesn’t quite work. In my experience, many organizations start by assigning it permission on behalf of the user who invoked it, which works for short, simple tasks but breaks down as they run longer or touch more systems. The next instinct is a service account, which solves delegation but creates over-permissioning and access that outlives its purpose. A more mature approach is to treat the agent as its own workload identity: short-lived, tightly scoped, ephemeral.

That’s the right target. But even a well-built workload identity assumes predictable behavior. It runs the code it was given, and that’s it. Agents don’t. An agent’s actual access can shift mid-task based on the prompt it received, the tool it decided to call, or the plugin it reached for. It has no fixed job role to provision against, and its lifecycle doesn’t align with the joiner-mover-leaver process built for human identities.

So even the mature version of workload identity gets you only part of the way there.

Why this is urgent now

In many enterprise environments, that’s already a structural problem, not a hypothetical one. Non-human identities already outnumber human ones, and that was true before agents showed up. Cloudflare has reported that automated traffic has overtaken human traffic in requests to the websites on its network, earlier than its own CEO had predicted.

That’s a measure of web requests rather than headcount, but the direction of travel is the same one I see inside the enterprise. Agents bend the curve upward because they can request their own tokens, call other services and spin up activity at machine speed. It gets more complicated once agents delegate to each other, a parent agent handing part of a task to several child agents, each inheriting a slice of permission from the one above it. A few layers deep, that’s a permission chain that may evolve beyond what any individual approver originally contemplated.

This growth is what turns the category problem from an interesting edge case into an operational one. Part of what makes this hard: non-human identities have never felt real to people the way human ones do. A human identity has a face. You can track down the person, ask why they need a given level of access and get a straight answer about their job.

A service account or an agent typically gives you none of that. It’s easy to leave alone until it’s compromised, and then you’re reconstructing what it was for and what it could reach. We tend to underestimate how many of these we have, and we underestimate what they can touch. That’s part of why the category problem went unaddressed for as long as it did. It’s not urgent until you can put a number on it, and the number is growing fast.

One CIO.com contributor recently argued the real question with agents is authority, the judgment an agent is allowed to exercise on the company’s behalf, not just access. I agree, but in my experience, many organizations aren’t yet able to answer the authority question because they’re still working out where these identities fit within existing governance frameworks.

The gap shows up in the same place most times I look for it: governance.

Where today’s identity programs break

In my experience, the failure point is rarely authentication. Increasingly, security teams have a handle on phishing-resistant multi-factor authentication, least privilege and continuous verification, or at least have them on the roadmap. Governance is the harder capability, and it’s where I see programs stall, and it’s where the category problem actually shows up day to day.

Visibility tells you what access exists. Ownership tells you who to call about it. Governance is what you do with that information, and it’s a different thing entirely: blocking risky access combinations before they’re granted, raising the approval bar automatically when a request is high-risk, catching and unwinding out-of-bounds access without waiting for a quarterly review to surface it.

What I’ve come to believe is that governance lags for reasons that have very little to do with technology. It needs executive backing, agreement across teams that don’t report to you and a willingness to change processes people would rather leave alone. Application owners have their own deadlines, and teams resist central controls that slow them down. You can buy a tool. You can’t buy the alignment, and that’s the part that stalls. Too often, agents make it worse, because they push a flood of identities that don’t fit your existing categories through a governance process that was already your weakest link.

Where to start

I don’t think waiting for the tooling to mature is an option because the agents are already here. Here’s the order I’d work in.

1. Build the foundation before you add complexity. Clean directories, enforced least privilege, offboarding that actually fires. Jumping to sophisticated continuous verification before those basics hold up just gives you a more elaborate version of the same gaps.
2. Know when to rebuild. When you design access models and policies, the instinct is to mirror what exists. That approach can reproduce years of accumulated permission creep into the new system. Start from least privilege and work up, and be willing to push on “we’ve always done it this way.”
3. Inventory your non-human identities now. Before agent deployments grow that footprint further, know what you have, who owns each one and what it’s allowed to do. This is more a governance problem than a technical one. Discovery is the hard part here; even mature tooling can struggle to give full visibility into NHIs, and that gap isn’t closing as fast as the agent count is growing.
4. Treat MFA as a floor. If your organization leans heavily on SMS or push-based authentication, build a path toward phishing-resistant methods. Attackers worked out the common ones long ago. And in the agentic era, some agentic systems can interact with authentication workflows on a user’s behalf, so a hijacked session token doesn’t just expose one account; it inherits that user’s full automated reach. Phishing-resistant architecture now means securing the token supply chain, not just passwords.
5. Assume credentials will be compromised. The useful question isn’t whether, it’s how much damage one stolen credential can do. Least privilege, segmentation, RBAC, short credential life spans and continuous monitoring are what limit the blast radius.

You’ll notice none of these are agent-specific, and for good reason. The discipline that governs agents is the same discipline that runs the rest of your identity program. Agents just take away the option of putting it off.

The administrative model isn’t enough anymore

For many human identities, or even traditional workflow identities, the governing question has historically been administrative: is this identity who it claims to be, and does it have permission to be here? In many environments, that question has traditionally been evaluated primarily at login, at provisioning, and it holds until the next review.

That approach can break down for an agent. Because an agent’s actual access scope can shift dynamically based on the prompt it receives (from a human or another agent), the tools it calls or the plugin it reaches for, knowing it authenticated successfully isn’t enough. That model was built for an identity whose attributes and permissions generally remain stable after authentication.

That’s the piece that has to be new, on top of the workload foundation. Identity governance should likely extend from something checked periodically into something that watches what the identity is doing, in real time, and flags the moment it drifts from what it was built for.

The teams that start building that layer now, while their agent count is manageable, are the ones who won’t be doing it in a hurry later.

A fraud model looks at an insurance claim and returns a score of 0.23: low risk. A SHAP explanation lays out exactly why: no prior claims, a modest claim value, an unremarkable claimant profile. A human adjuster reads the explanation, agrees with it and signs off. Every box that explainable AI asks us to check has been checked. The claim is settled and closed.

By any current standard for responsible AI, this is exactly what a good outcome looks like.

Two months later, an audit

Two months after the claim closes, it’s pulled into a routine quarterly audit sample (no suspicion attached, just standard sampling). The auditor isn’t reviewing the model’s reasoning. She’s asking a narrower question: can this decision be reconstructed, independently, from its own evidence trail?

She doesn’t start with the fraud score. She starts with the paperwork.

Her first question: what established the date of the incident? She pulls the document meant to corroborate it and checks something the claim file itself never surfaces, the document’s own creation metadata. It was generated two days after the claim was filed. She asks the obvious follow-up: did anyone check this before the claim reached the model? Nobody had. Document provenance was never captured anywhere in the intake pipeline as a fact to be verified; the claim reached the model looking exactly like every clean claim before it.

Knowledge gap: This doesn’t establish that the claim is fraudulent. A prior version of the document may exist and say the same thing, in which case the late timestamp is a non-event. What it establishes is that the system has no way to answer that question either way. Document lineage was never captured, so there’s no way to know whether an earlier version existed, or whether it agreed with this one.

Her second question: when was the insured asset last valued? The figure the model used is 22 months old relative to the date of loss. Nothing about it is wrong or falsified; it’s simply the most recent valuation on file, resolved to the wrong point in time and handed to the model as though it were current.

Knowledge gap: This doesn’t establish that the valuation is wrong. The asset’s value may not have moved materially in 22 months, in which case the old figure and a fresh one would say the same thing. What it establishes is that the system has no process for knowing which is true. There’s no trigger for re-valuation and no record of whether the figure was ever checked against current conditions.

Her third question: as of what date was the claimant’s risk profile evaluated? The “current risk” features the model consumed were pulled at query time (the day the claim happened to be scored) rather than resolved back to the date of loss. She asks whether that was a deliberate design choice. Nobody can tell her. There’s no rule on record, no document, no named owner for the decision “risk features should reflect the date of loss, not the day the system happens to be queried.” It’s simply how the pipeline was built.

Policy version gap: This doesn’t establish that the score would have come out differently under incident-time data. The claimant’s risk profile may not have changed since the date of loss, in which case query-time and incident-time features would produce the same result. What it establishes is that nobody decided which point in time should govern the decision, so there’s no way to know whether the two agree, only that nobody checked.

Three questions. Three gaps. Not one of them touches the model, and not one of them would have shown up in a model audit, but all of them question the explainability of the decision taken.

The blind spot in explainability

This is the uncomfortable pattern showing up across banking, insurance, healthcare and government AI deployments: the model is explainable, the explanation is faithful and the decision is still wrong. It’s worth being precise about why, because the instinct is to reach for a better interpretability technique, or a smarter document-reading agent, and neither fixes this class of failure.

Every mainstream explainability approach shares one architectural trait, whether it’s SHAP, LIME, DARPA’s XAI program, the EU AI Act’s Article 13 disclosures, or IBM’s AI FactSheets: they all operate after the model has reasoned. They take the inputs as given, assumed correct and explain what the model did with them. That’s legitimate and valuable. It’s also, structurally, a blind spot for the class of failure the audit just surfaced.

An enterprise decision isn’t just a model output. It’s a reasoning process running over a knowledge base and a policy set, both of which have to be current, correctly sourced and resolved to the right moment in time before the model ever sees them. A model can reason perfectly over compromised inputs and produce a perfectly explainable, perfectly wrong decision. No amount of post-hoc transparency catches that, because post-hoc transparency was never looking at the inputs; it was only looking at what the model did with them.

Put precisely: a decision is explainable only when its reasoning, its knowledge and its policy are all explainable at the same time. Explainable AI delivers exactly one of those three: explainable reasoning. It can be flawless, and the decision can still be unexplainable overall, because an unverifiable knowledge input or a misapplied policy breaks the chain regardless of how good the reasoning layer is. That’s the precise sense in which explainable AI is necessary but not sufficient. Sufficiency is a property of the whole decision (knowledge, policy and reasoning together), not of any one component, however mature that component’s tooling has become.

Where the audit’s three questions actually point

Map the auditor’s three findings back onto that structure and the pattern sharpens.

Her first and second questions (the backdated document, the stale valuation) are both knowledge failures. Neither is a data-entry error; both are facts that were never traceable to an authoritative source, versioned and resolved to the right point in time before reaching the model.

Her third question (features pulled at query time rather than incident time) is a policy failure. No rule existed to say which moment in time the decision should be evaluated against, so the system defaulted to whatever was easiest to compute: now.

Governing the inputs, not just the output

The fix isn’t a better explanation of the model, and it isn’t a more sophisticated agent reading the documents more carefully. It’s governance that sits before the model, a framework I call Explainable Decision Systems (EDS), built on three pillars plus a temporal layer that ties them together.

This tracks a broader shift already visible across enterprise AI deployments: the stacks that win aren’t the ones that ship fastest, but the ones that get data and decisions governed before anything downstream runs.

• Explainable Knowledge (EK). Every fact fed into a decision has to be traceable to an authoritative source, versioned and (critically) resolved to the valid time of the event being decided, not the time someone happened to query the system or the date typed into a form field. An asset valuation from 22 months ago isn’t wrong data; it’s data resolved to the wrong point in time.
• Explainable Policy (EP). The policy version that governs a decision is the one that was active when the event occurred, not the one currently in the system. A claim from November has to be judged against the policy wording in force in November (exclusion schedule and all), retrieved and locked before reasoning starts, not looked up after the fact.
• Explainable Reasoning (ER). This is where existing XAI actually earns its keep, but EDS splits it into two accountable parts: AI reasoning (EAI), which needs interpretability techniques like SHAP, and Human Decision (HD) (overrides, approvals, escalations), which needs an identified actor, a timestamp and a documented justification. EDS doesn’t claim human judgment is formally explainable in the way a model’s math is; it just insists that it’s attributable and auditable.

The layer that binds all three together is bi-temporal governance: knowledge and policy have to resolve to the real-world moment the decision concerns, not the moment the system happens to be queried. In EDS, this resolution (called a temporal witness) isn’t a report generated after the decision, the way the audit in this piece was. It’s a gate the decision has to pass before reasoning is allowed to start. Had the witness been in place, the claim in this scenario would never have reached the model at all; it would have been flagged as a governance exception the moment the document’s creation timestamp failed to match its claimed valid time.

Why no AI agent can catch the hospital record

This is worth slowing down on, because it’s the clearest illustration of why EK has to be its own dimension, entirely separate from model explainability or agentic sophistication.

Every clinical or administrative record system has two different dates living in it. There’s the stated field value (“date of incident,” “date of visit”), which is just a form entry: whatever a person typed, told a clinician or was asked to record. And there’s the system-of-record creation timestamp: the moment the backend actually inserted that row, logged that transaction or wrote that file. That timestamp sits in the origin system’s audit trail, not on the document itself.

A claimant doesn’t need to forge anything to exploit the gap between those two dates. They need only have a record created on 30 November whose stated incident-date field reads “15 November.” The document is completely genuine. Nothing about it is altered, and no forensic check (pixel analysis, formatting consistency, metadata inspection of the file itself) will find anything wrong, because there is nothing wrong with the document. The lie isn’t in the artifact. It’s in the mismatch between what the artifact claims and when an entirely separate system actually created it.

Hand that document to an AI agent, no matter how advanced, and ask it to check for fraud. It will read the stated date, assess whether the narrative is internally consistent, run every forensic technique available and come back clean; it’s answering the wrong question. It’s checking whether the document is coherent and unaltered. The question that actually matters is whether the document’s claimed date matches the creation-time metadata logged by the system that produced it. That answer doesn’t live in the document. It lives in a separate system, under separate access control and requires a deliberate governance step to retrieve and cross-reference. No amount of model capability substitutes for that step, because the information the model would need simply isn’t present in what it’s being asked to read.

This is exactly the gap enterprise surveys keep flagging. In Grant Thornton’s 2026 AI Impact Survey of nearly 1,000 senior business leaders, 78% said they lacked confidence their organization could pass an independent AI governance audit within ninety days, precisely because most are scaling AI they cannot explain, measure or defend. That’s the downstream version of the same problem this claim ran into: explainability treated as a reporting exercise rather than a precondition the case has to clear before it ever reaches the model.

That is precisely what Explainable Knowledge formalizes: a requirement that every input be provenance-linked to its origin system, with independently retrieved creation-time metadata checked against the claimed valid time; not a soft appeal to “make sure your data is good,” but a specific, checkable technical control. Under EDS, if that cross-reference can’t be performed, the claim doesn’t proceed to the fraud model. It’s blocked as a governance exception, regardless of how convincing the document looks or how confidently any reader, human or AI, assesses it.

These aren’t academic distinctions, either. In the UK and international context, failures like these map to live regulatory obligations: FCA claims-handling rules on treating customers fairly, Federal Reserve SR 11-7 guidance on model risk management, and the EU AI Act’s Article 9 risk-management requirements for high-risk systems. Regulators are already asking questions that XAI, and the agents built on top of it, cannot answer.

What this means for the people building these systems

For enterprise architects, the practical shift is this: stop treating knowledge and policy governance as documentation you produce after a model runs (or worse, as something an audit uncovers two months later) and start treating it as a gate the model has to pass through before it runs. This is the same direction major risk-consulting practices are now pushing clients toward: governance embedded as a control layer across the AI lifecycle, rather than a compliance sign-off at the end of it. Concretely, that means:

• Bi-temporal data stores for anything that feeds a decision, so “valid at the time of the event” is a queryable property, not an assumption.
• Provenance links from every document or claim to its origin system’s own creation-time metadata, captured independently of the stated field values on the document (not derived from them).
• Policy version registries with effective and expiry dates, so the version applied is probably the one that was active when the event occurred.
• A pre-inference middleware layer that checks provenance, versioning and temporal validity before forwarding anything to the model, and blocks or flags the case if it can’t.
• Explicit, attributable records for human overrides: who, when, under what authority and why.

None of this replaces SHAP, LIME or your existing model-risk documentation, and it isn’t a call for a smarter document-reading agent either. It sits in front of both, deciding whether they’re even looking at trustworthy, independently verified inputs.

The bar has moved

Explainability was always going to be necessary. Regulators demand it, customers expect it and no serious enterprise AI deployment should ship without it. But “the model is explainable” and “the decision is governable” have quietly become two different claims, and treating them as one is how a technically pristine 0.23 fraud score sails through, gets closed and sits untouched for two months until an auditor asks the one question explainability was never built to answer: not what did the model do, but what was it allowed to look at, and when, and against what independent record?

Architectures that can’t answer that question will keep producing decisions that are perfectly explainable and quietly wrong, right up until the audit finds them.

Let’s say one of your teams builds an AI agent that actually works. Word gets around, and seemingly overnight, there are twenty more built by people in finance, legal, HR, and support. Most of them are useful, but when someone suddenly gets a chatbot response showing customer data they shouldn’t have access to, reality hits.

The real test of enterprise AI readiness isn’t at all whether your coworkers can confidently work with AI. Instead, it has everything to do with governance and security, global, cross-cutting policy, and privacy. Many enterprises are underprepared to face these issues: a survey from Databricks and the Economist found that “40% of respondents believed their organization’s AI governance program is insufficient.” And Microsoft’s Data Security Index reports that “only 47% of organizations across industries report they are implementing specific GenAI security controls.”

Having worked with many CIOs to develop strategies to govern their AI systems, this piece is a deep dive into the specifics of what works and what doesn’t. Keeping employees, customers and your entire organization safe must be your top priority before you even start rolling agents out.

8 layers of governance every enterprise needs

It’s no doubt that the enthusiasm for AI is real, but so is the list of questions that bubble up a month later: Who’s allowed to publish an agent to the rest of the company? How do we track versions, and can we roll one back? Can we require SSO on every agent? What data does this have access to, and does it respect the permissions on those documents? Where do the logs live, and how quickly can we see them?

I think of enterprise AI governance as a set of layers, each answering one of the questions above. You can build them incrementally, but ideally all eight are in place before you have more agents than you can list off from memory.

1. Roles and groups. Whatever platform you choose, ensure that it offers granular role-based access control. At the level of abstraction higher, map groups to real departments (Legal, HR, Capture Team) and assign those groups to their own. Keep the admin count small enough to list out loud. Though this is the coarsest measure, it still bears a big responsibility down the line — according to Fortune Business Insights, “RBAC solutions help reduce unauthorized access incidents by nearly 30%, enhancing data security.”
2. Scope. It’s extremely helpful to build out private folders with explicit allowlists, so a project, agent, or workflow isn’t merely locked to outsiders but invisible to them. Agents with access to sensitive customer data shouldn’t advertise their own existence.
3. Change control. Those who are building AI agents should be able to place locks on so that only the owner can edit it, with admin override. Every change should land in a version history with a diff and a commit message, and rolling back should take one click so that you know exactly what was edited if an agent stops performing. According to Google’s DORA State of DevOps research, teams with strong version control and rollback practices recover from failed changes in under an hour, versus a week or more for teams without them.
4. Publication. When a builder finalizes an agent and wants to release it to the rest of their team, depending on the use case, they should consider adding one-click SSO, a password option for external collaborators, and restrictions on which origins and which users can reach it. OWASP’s 2025 Top 10 keeps broken access control at #1, observing that every application they tested had some form of broken access control.
5. Org-wide policy. This is the layer CIOs underuse. Require SSO on all interfaces rather than hoping. Restrict who can publish, so shipping to the company is an admin action. Create approval workflows, where a builder requests review and an admin does the publishing (similar to traditional SDLC best practices around pull requests). Allow or deny specific tools and connectors across the enterprise, so nobody connects a data source that procurement hasn’t approved yet. Lastly, vary policy by group, so only Legal can reach the legal agents.
6. Data access. Obsidian Security’s 2025 AI Agent Security Landscape report found that 90% of deployed AI agents are over-permissioned relative to the actual scope of their assigned tasks, and separately estimates that agents are typically granted about 10 times more access than their workflows need. That’s why connections and knowledge bases deserve their own permission model. Credentials should be encrypted and owned by whoever created the connection to a given tool, app, or data lake, with sharing as an explicit decision. Here’s where it gets tricky: if a builder connects SharePoint with their own account, the agent sees what that person can see. Connect through a service account and the agent inherits the service account’s entire scope, which is usually far wider than anyone intended. The safer pattern is to check end-user access at runtime, so the person asking has to authenticate before retrieval happens and only gets what they’re already entitled to.
7. Observability. It’s imperative to have exportable records of who ran what, when, against which model, with token counts and latency, plus per-step traces showing inputs, retrieved chunks and outputs. Give builders the ability to mask or disable logging where the data is too sensitive to retain. Then push those records out of the platform on a schedule so security’s pipeline consumes them automatically.
8. Authentication. SSO over passwords, MFA through whatever authenticator your org already runs, and defaulting new users into the lowest-privilege role until an admin promotes them. According to Microsoft, MFA can block more than 99.2% of account compromise attacks. And SSO gives you the email address of everyone using your interfaces, which makes for an additional layer of security.

None of these are new inventions; they’re the same controls you already apply to internal software. But most organizations haven’t extended them to AI that both talks to customers and writes to the CRM.

Don’t forget about deployment

After governance, the other half of enterprise AI readiness is deployment. Regardless of what platform you choose, there are four realistic postures:

1. Multi-tenant SaaS, where you share infrastructure with others and rely on tenant isolation at the database layer.
2. Dedicated single-tenant, where the vendor operates a VPC that only you occupy.
3. Bring-your-own-cloud, where the software runs inside your own cloud account and your data never leaves it.
4. On-premise, where you own the infrastructure, the control plane and the data plane outright.

It’s well known that multi-tenant is available as soon as possible and updates itself continuously. Dedicated takes a couple of weeks to stand up and updates on a schedule the vendor applies. Bring-your-own-cloud runs two to four weeks and splits responsibility, with the vendor managing the application through a scoped cross-account role while your team owns networking and IAM. On-premise starts at a month or more, and after that your team applies every release itself.

My honest read? Most organizations are well served by multi-tenant, and choosing it doesn’t compromise on security or privacy. Bring-your-own-cloud and on-premise make sense when you have a data sovereignty requirement and a mature internal platform team, but they result in manual updates and slower time to value. There’s no right or wrong answer here — but it’s helpful for CIOs to understand the trade-offs between each of these common deployment methods so they can choose what fits best for their enterprise.

A readiness test you can run this week

So, are you ready for enterprise AI? Here’s a quick self-diagnostic that I like to run with CIOs. Pick your three highest-privilege AI agents or workflows in production and try to list these attributes yourself:

Who owns each one by name. What changed in it most recently, and who approved that change. Which connectors and knowledge bases it can reach, and whose permissions those reads run under. Whether accessing it requires SSO. Where its logs are, and how long they’re retained. What happens to it if the model version it depends on is deprecated next quarter.

If most of those answers are “I’d have to ask around,” you’re not ready to scale, and that’s fine. All it means is that you have a week of policy work ahead — but you’ll thank yourself later for figuring the tough stuff out first.

For most of the IT era, technology leaders have treated information as an asset. It is something to be stored, secured, processed and monetized. Organizations generated vast amounts of data, and technology processed that wealth and made it useful.

Then AI arrived, and the framing broke.

AI changes the relationship with information. It interprets it, compresses patterns within it, and generates new language from those patterns. It does so with such fluency that it converts accumulated human expression into outputs that are coherent, meaningful, and often persuasive in ways that are difficult to examine or trace. The arrival of AI is a deeper shift in how organizations produce language, remember knowledge, establish authenticity and decide what deserves trust.

Generally, the discussion about AI’s impact on language, memory and meaning gets swept into the social or philosophical bucket too quickly. But these are not just philosophical questions. They are enterprise questions, because they directly affect knowledge management, brand trust, customer engagement, regulatory exposure, institutional memory, decision-making and employee learning. CIOs who brush this dimension aside will govern AI infrastructure competently and miss its deeper institutional consequences entirely.

Language is humanity’s most powerful mechanism for collective learning. We speak not only about what is present, but about what is absent, imagined, remembered, feared and hoped for. That capacity is what allowed us to preserve experience, transmit it across generations and convert it into culture, knowledge, law, philosophy, science and enterprise memory.

CIOs must govern meaning, not just data

This is why large language models are consequential in a way that earlier software was not. LLMs train on enormous volumes of text. They learn patterns, structures, associations, idioms and contextual relationships within language. By doing this, they create a dynamic simulation of human language itself. Unlike traditional software that executes defined instructions, LLMs operate within encoded human expression. They observe documents, reports, messages and public knowledge, and they generate responses that sound almost like understanding.

Almost. And that almost is where the situation gets hairy.

The power is real. AI can widen access to sophisticated knowledge. It can digest complexity, translate across domains and reveal patterns invisible to a single analyst. Inside an organization, the impact can be significant: a junior employee can consume decades of internal documents in minutes; a manager can synthesise thousands of customer interactions before a single meeting; a compliance team can identify patterns across audit data that would take months by hand. Knowledge work has a new definition.

But it is precisely this power that creates the crisis of synthetic culture.

While writing “The AI Codex: Power, Ethics and the Human Future in the Age of Intelligent Machines,” I chose to leave the engineering largely aside and follow the subtler human consequences that are easier to ignore.

Synthetic culture is one of those consequences, and it should concern CIOs immensely.

Culture is not office decoration, value posters or elaborate town halls. Culture is how an organization thinks, decides, explains, rewards and justifies its actions. It lives not only in documents but in stories, habits, unwritten rules, leadership behaviours, institutional scars and the memory of battles won or lost. It is what people have lived through and passed on. Your document management system and knowledge management system are not the custodian of your culture. The people around you are.

AI preserves the digital residue of culture. It cannot preserve the lived meaning behind it. That distinction matters enormously.

Consider what happens in practice. If AI generates a memo in the style of a CEO, does it carry the judgment of that leader, or does it merely carry the pattern of her language? When generative AI summarises a complex customer dispute, does it carry the flavour of the relational history, or does it just compress text? When it drafts a new policy, does it reflect institutional accountability, or does it mimic the average structure of similar policies? When AI produces a cultural narrative for employees, is it transmitting memory or manufacturing a convincing imitation of the same?

These are not rhetorical questions. They describe a genuine ambiguity that is already embedded in enterprise operations.

AI takes what is human, learns from it, and produces something that comes close to human expression. But proximity is not identity. AI can sound human without being human. It can produce language without emotion. It can generate meaningful output without moral agency. It can create efficiency without bearing responsibility for the consequences.

And this is where authenticity begins to fracture.

There was, until recently, something like a one-to-one relationship between content and its source. A document had an author. A speech had a speaker. A photograph recorded an event. A policy had an accountable authority behind it. These relationships were not perfect, but they were there, and they allowed organizations and societies to trace meaning back to a human being who could be questioned, challenged or held responsible.

AI can weaken or obscure that link. It can produce synthetic truths that spread rapidly, appear credible and influence decisions without being anchored to any lived reality. Deepfakes are the obvious example. But the real danger is subtler and already present inside organizations: the synthetic summary, the automated narrative, the AI-generated recommendation that nobody can fully trace. A synthetic truth is not a lie. But it can be more dangerous than one, because it poses as a plausible construction with no real thing behind it.

The new enterprise risk: synthetic truth

For CIOs, this is a new category of risk for which many organizations have no mature controls. The question is no longer only whether organizational data is secure. The question is also whether organizational meaning is secure. Can employees distinguish between original knowledge and synthetic synthesis? Can customers trust that they are interacting with accountable institutional communications, or with an automated approximation? Can leadership trace the lineage of a recommendation or a decision?

These are operational questions. The failure mode is not just a data breach but more of a memory breach.

When AI systems generate from enterprise knowledge, they shape what the organization remembers and how it remembers it.

AI is changing what organizations remember

The consequences follow from the quality of what goes in. If the underlying data reflects poor documentation, AI amplifies that poverty. If institutional knowledge records only dominant voices, dissenting experience is subdued and eventually forgotten. If past mistakes have been quietly buried, AI may reproduce the organization’s confidence without preserving its caution. The organization becomes more efficient at forgetting what it should have remembered.

CIOs also need to rethink what knowledge management means. For years, KM was treated as a repository problem: store, tag, search, retrieve. AI retrieves knowledge and generates new formulations from it. Each time a model runs, it can produce a slightly different answer. Knowledge becomes fluid and unstable. A document may be old, but it is static and accountable. An AI-generated answer may be elegant but untraceable. Both can exist in the same organization, and most people cannot tell them apart.

Those who complained about information overload in the internet age have no idea about the blizzard the AI age is about to bring. AI can identify patterns invisible to humans, but it can also manufacture alternative truths that are difficult to challenge. It can reduce noise, but it can also generate noise at industrial scale.

This is why AI governance cannot be a downstream compliance exercise. It must be a first-principle commitment. Not just checking whether the model works but asking what kind of institutional memory the model is helping to create. Where did the information come from? Who approved its use? What has been included, and more importantly, what has been excluded? Where does the audit trail begin? Where does human judgment remain mandatory and non-negotiable? These questions belong on the Post-it notes sitting on every CIO’s desk as the AI agenda gathers speed.

The search for truth cannot be a human pursuit alone in this environment, but it cannot be outsourced to machines either. It must be a governed exercise, with explicit architecture and explicit accountability.

The crisis of synthetic culture will not announce itself dramatically. It will arrive quietly, in the form of convenience: automated memos, summarised knowledge, AI-generated reports that nobody has the time or the inclination to question. Machines will not become human. But organizations will gradually grow comfortable accepting machine-generated statistical approximations as human judgment, institutional memory or cultural truth. That comfort is the real risk.

The CIO now carries an institutional mandate: scale intelligence without surrendering trust. Productivity matters and must be pursued. But the real challenge is building trust while scaling intelligence. This means ensuring AI output is traceable, sources are visible, human authority is explicit and institutional memory is always protected from synthetic distortion.

CIOs have to become not just custodians of organizational systems, but of organizational memory. In the new Badlands of the AI age, the CIO is the morally upright gunslinger. Organizational memory is the line she must defend.

A few months ago, I was sitting in a glass-walled conference room with the executive team of a fast-growing enterprise. The vice president of customer operations was enthusiastically demonstrating the new automated agent features their software vendor had just pushed into their CRM platform.

On the screen, the software looked brilliant. The agent could read customer complaints, analyze transaction histories and automatically resolve issues. The VP showed us how the system could independently offer retention incentives to unhappy accounts without a human ever touching a keyboard.

Then I asked a simple question: “What is your approval process when the AI decides to grant a $20,000 contract discount to keep a customer from leaving?”

The room went completely silent. The VP looked at the director of IT, the director of IT looked at the chief risk officer, and everyone realized the same thing at the exact same moment. They had spent three months evaluating software licenses and security protocols, but nobody had asked who gave the software permission to sign off on corporate spending.

Major software providers like Salesforce, SAP and Oracle are rapidly moving beyond simple report writers and conversational chatbots. They are embedding active, autonomous agents directly into the transactional core of systems that manage your revenue, customer agreements and financial ledgers. According to Gartner’s latest adoption forecasts, eighty percent of enterprise applications will deploy these embedded capabilities by 2026. These applications do not just summarize data: they issue refunds, alter contract terms and trigger supply chain orders.

When I review these deployments with client teams, the core problem has nothing to do with artificial intelligence. It is a fundamental breakdown in corporate delegation and signing authority.

The breakdown of the corporate signing matrix

Every mature company I work with operates on a clear delegation of authority matrix. This framework dictates exactly who can sign off on financial commitments. A vice president might have authorization to approve spending up to $500,000, a director might sit at $100,000 and a front-line manager might be capped at $500. For two decades, technology leaders have spent millions of dollars building security and compliance controls to ensure every human employee operates strictly within those limits.

Yet when a software vendor releases an update featuring autonomous agents, companies routinely grant these features unrestricted operational freedom. Because the capability arrives as a native feature inside an existing application, business units enable it with a single click. In my advisory work, I repeatedly see organizations grant third-party software features more financial freedom than their own human managers.

This represents a massive blind spot in executive governance. McKinsey’s global surveys on artificial intelligence reveal a striking pattern across the enterprise landscape: while adoption is accelerating at a historic pace, only a tiny fraction of organizations are actively managing the financial and operational risks of automated decision errors.

The quiet cost of shadow delegation

In my audits, this rarely manifests as a dramatic system crash. It plays out as a quiet margin leak. In one organization I reviewed, a department head had enabled an automated customer retention feature over a weekend. The agent noticed an important account expressing frustration in a support ticket, and to prevent the account from churning, it independently applied an unapproved 15 percent discount to their multi-year contract.

The customer was happy, and the account manager considered the client saved. But from an executive perspective, an unvetted third-party algorithm just executed an unauthorized contract modification that eroded company margins. When the finance team conducted a quarterly audit, they did not discover an employee violating spending policy. They discovered a black-box automated decision that bypassed every internal approval control in the company.

When an auditor tests your internal controls, presenting a log showing that a vendor’s algorithm made an unauthorized financial change does not satisfy the requirement. If an action requires managerial sign-off when performed by a human being, letting software execute it independently is a major control failure.

How I advise executive teams to handle automated authority

Protecting your organization does not mean turning off these tools or falling behind on technology. It means treating vendor-supplied agents exactly like third-party contractors who have not yet passed a background check.

Forrester Research emphasizes that extending zero-trust security frameworks to automated business processes is now mandatory for enterprise risk management. Zero-trust simply means that no user, device or automated tool gets implicit trust. Every proposed action must be validated against explicit business rules before it happens.

When I help enterprise teams design these safeguards, we establish a practical three-tiered boundary for automated tools:

• Read and draft permission: Automated tools can freely analyze trends, draft emails and assemble internal reports. No human sign-off is needed to create a draft, but the system cannot publish or execute anything on its own.
• Standard administrative permission: Tools can handle routine administrative tasks or process standard requests below a strict financial cap (such as a $50 service credit), provided every single action is logged in an audit file that managers review weekly.
• Restricted financial permission: Any action that alters contract terms, changes pricing tiers or issues major refunds are strictly held in an authorization queue. The system generates the request, but a human manager must click “approve” before the change hits the live database.

As a technology executive, you cannot control what automated features software providers bundle into their platforms. You can, however, control the financial boundaries and signing authority those tools are permitted to exercise within your business.

What to do at your next executive leadership meeting

1. Ask for an automated authority inventory: Have your team audit your core software platforms to identify every automated feature currently running with permission to alter financial or customer records.
2. Revert to draft-only mode: Instruct your team to default all vendor-supplied automated agents to “draft only” until a clear business case justifies giving them independent operational authority.
3. Establish a firm human-in-the-loop rule: Require a strict organizational policy that no automated system can modify pricing, contracts or financial ledgers without explicit manager approval.

Speed-to-market dominates enterprise AI priorities in 2026. Beyond upfront resourcing costs of prioritizing speed, organizations face a more insidious risk:  the compounding cost of ungoverned AI.

In November 2019, a tech entrepreneur signing up for the newly launched Apple Card publicly complained that he received a credit limit 20 times higher than his wife’s, despite joint tax filings and her higher credit score. Steve Wozniak had a similar experience: a limit 10 times higher than his wife’s. Retrospectively, these revelations were the canary in the coal mine.

In the years that followed, Apple and its credit partner, Goldman Sachs, drew legal and regulatory scrutiny over gender bias and consumer protection issues. The CFPB’s 2024 order documented that Apple had forced Goldman Sachs to accelerate deployment by attaching a $25 million penalty to every 90-day launch delay. 

Prioritizing launch speed — ship first, address problems later — over building a functioning disputes process created years of cascading failures. Apple and Goldman Sachs were ordered to pay $89 million in penalties and consumer redress. Prohibited from launching another credit card until it could demonstrate a credible plan to comply with the law, Goldman Sachs lost money on Apple Card for years and ultimately sold its consumer credit line. The legal and compliance penalties were only a fraction of the total costs.

If a deterministic underwriting system can create liability at this scale, the risks posed by agentic AI are substantially greater:  autonomous systems can multiply and scale errors, quietly and invisibly, at machine speed.

The accelerated cost of ungoverned speed

In software engineering, shortcuts taken to ship are called “technical debt.” When teams sacrifice robust architecture, processes or solutions to reach deadlines, interest accrues in the codebase as brittle integrations and expensive refactoring. When it comes to agentic AI, technical debt accrues faster. AI portfolio returns are estimated to drop by 18% to 29% when technical debt is ignored.

Similar to technical debt, AI governance debt accumulates when speed-to-market routinely takes precedence. Unlike technical debt, which can wait silently in a repository without immediate consequence, governance debt is neither patient nor pausable. Deployed without clear authorizations, boundaries, or constraints, AI systems scale defects across an enterprise at machine speed.

The agentic multiplier: Scaling decisions and outcomes

When a standard LLM produces an output, a person receives it and decides what to do with it. That pause is a crucial point of control:  a “human gate” stands between the generative AI model and the consequence. AI agents operate in continuous, (semi-)autonomous loops, without human gate friction. With agents, a model error can cascade downstream unimpeded through enterprise systems. Multi-agent systems inadequately governed have error rates of nearly 20%. Generative AI scales outputs, and agentic AI scales outcomes. Put another way, agentic AI scales outcome-producing actions, and every autonomous action carries a decision that a human used to make. 

AI governance is often mischaracterized as “putting the brakes on” speed-to-market. In practice, omitting it causes “velocity decay.” While 54% of leaders consider governance to be an obstacle to scaling, its absence or inadequacy creates sociotechnical bottlenecks that ultimately stall deployment and operations. To prevent both velocity decay and governance debt, governance must “shift left” to be architected throughout the AI system lifecycle.

Accountability reduces the debt

Technical accountability within an autonomous system cannot exist in a vacuum; it requires both structural and cultural accountability throughout an organization.

Structural accountability

Structural accountability assigns formal ownership over agent actions to specific human decision-makers. A July 2026 white paper, Safeguards for Agentic Finance at Runtime (SAFR), provides specific case studies from the financial services industry that include and advocate for structured human accountability.

• Individual ownership of agent decisions:  Any decisions made by an agent that are highly consequential require a human owner. Think of a CFO signing her name to financial statements: if auditors or regulators take issue with those documents, she is named as responsible for the numbers. Her successor inherits the same accountability. Consequential decisions, such as a denied insurance claim, a moved financial asset or a rejected job applicant, require a human owner and a process to identify them (detailed below).
• Shared ownership of agent outcomes:  When an autonomous workflow crosses traditional siloes, such as a logistics agent altering supply chain routes based on real-time marketing data, ownership becomes distributed. While externally, an organization remains a single liable entity,  internally, it must assign accountability across organizational boundaries. Mechanisms like a Joint Accountability Agreement (JAA) can facilitate this by explicitly aligning cross-functional decision rights, escalation protocols and continuous monitoring metrics. 

Cultural accountability

Cultural accountability means everyone has a role to play, and everyone owns both the final result and the process to get there. Think of a crew team rowing: everyone rows to win, and everyone is responsible for both individual performance (like erg times) and the team’s overall success (race speed and ranking). High-ownership cultures ensure that accountable behaviors are recognized and rewarded, and visible consequences exist when accountability is lacking. Just as accountability is instilled in a crew team through clear, shared goals and transparency on individual and team effort, metrics and results, employees can be incentivized to own individual and collective actions, outputs and outcomes. Importantly, these accountable behaviors enable accountable AI. If these employees are also empowered to challenge AI, they are equally empowered to own its results. Knowing they will be rewarded or recognized for interceding – not punished – is critical to reduce governance debt and prevent velocity decay.

Human accountability in the agent workflow

Product Advisory Collective

The diagram maps structural and cultural human accountability with an agent’s workflow:  monitor context, make decisions, coordinate, complete tasks, and deliver an outcome. When decision-making is consequential, a single human owner must be accountable (Step 2), whereas shared ownership of agent outcomes, across all of the teams and individuals that contributed to the workflow or are impacted by its results (Step 5), is typically necessary. Underpinning all five steps is cultural accountability, which facilitates accountable human behaviors and enables human workers to detect and prevent unaccountable agent behaviors.

Importantly, whether an agent’s decisions (Step 2) require a single named owner depends on the severity of consequences for the enterprise and its key stakeholders. A named owner signs off on the risk criteria and thresholds, and answers for any consequences that occur if risk thresholds are surpassed. The SAFR white paper recommends evaluating five risk criteria: action reversibility, financial materiality, customer impact severity, regulatory sensitivity and novelty or anomaly. Risk thresholds are set pre-deployment, and proposed agent actions (Step 2) are evaluated continuously at runtime. While “above threshold” risks trigger a real-time human-in-the-loop (HITL) review, the executive owner remains ultimately responsible for any resulting repercussions or systemic issues.

Too often, model decision-making oversight is lacking. While agent orchestration and escalations to HITL reviewers are established practices, scrutiny over the risk or consequences of the model’s decisions is the exception, not the rule, even in high-risk industries. As recently as 2023, roughly 40% of hospital systems did not evaluate AI models for accuracy and 56% did not evaluate them for bias. This clear lack of oversight is governance debt with acute liability. Preventing and reducing the debt requires formal structural ownership and a culture that rewards accountability. 

Strategic recommendations

A unique organizational impact of agentic AI is that it collapses traditional operational boundaries, like safety and security, converging performance metrics, systemic risks and multi-jurisdictional compliance requirements over time. To address this convergence of performance and governance issues, we recommend systemic interventions, including that leadership:

• Streamline governance efforts strategically to account for the collapsed operational boundaries and eliminate organizational friction and bottlenecks.
• Structure accountability into roles:  assign names to consequential decisions and set up JAAs to manage shared accountability.
• Shift governance left by embedding oversight early and continuously across the system lifecycle.
• Design governance ex ante, not ex post: consider it a core architectural and infrastructural requirement, rather than post-launch remediation.
• Scale efficiently with policy-as-code: supplement or replace siloed and unnecessary manual oversight with automated computational controls.

Final takeaways

For CIOs, the ultimate mandate is to anchor AI governance directly in day-to-day infrastructure, roles and responsibilities. When autonomous agents scale risks instantly, traditional manual reviews and human-controlled workflows simply cannot keep pace. Oversight must “shift left” and be strategically built into system logic from the start.

Unchecked speed-to-market is an expensive illusion. As Apple Card’s costly errors and losses demonstrate, a rushed launch and a lack of oversight can result in financial penalties, remediation and loss of trust, reputation and business. To avoid the repercussions – including the velocity decay – that accompany governance debt, leaders must proactively architect accountability across the enterprise, the system lifecycle and critical decisions.

Eliminating systemic blind spots doesn’t require perfect foresight; it requires intentional architecture, well-designed collaborative ownership across silos and explicitly named human decision-makers. It’s possible to prevent the compounding liability that falls between AI decision-making and unstructured human accountability by answering two key questions early on:  Who owns which decisions? And how do we clearly incentivize accountability?

Enterprises have built their data systems for humans, but AI agents need a whole new infrastructure.

Separate research from Cloudera and Google/MIT found that, not surprisingly, there is fervent enterprise interest in AI agents, but underlying infrastructure struggles to keep up. Deployments continue to be hampered, sometimes even abandoned, largely due to issues with data access, context, and governance.

“Enterprise adoption of agentic AI is on the cusp of an extraordinary acceleration,” the Google/MIT report noted. “As organizations look to scale agentic AI across the enterprise, they cannot ignore their data systems.”

Resolving data bottlenecks, then, should be an immediate priority.

Projects delayed, inaccessible data

Cloudera’s report, created in partnership with Wakefield Research, describes the need for a “great AI re-architecture.”

Of the 1,500 enterprise architects and cloud infrastructure leads surveyed, a stunning 95% said they had delayed or cancelled AI projects, in some cases six or more, in the past year, due to issues with data governance, compliance, or regulatory issues.

A wide majority also reported that AI integrations have changed their data storage and architecture practices, AI workloads have increased infrastructure costs, and current data architecture requires a “significant overhaul” to meet AI goals.

“Even if enterprises are ready to use AI, many are coming to the realization that the foundational infrastructure it relies on is not,” the report noted.

Similarly, more than half of the 300 IT execs and heads of product, IT, data, and AI responding to the Google/MIT survey said they have paused or delayed the deployment of AI agents to address foundational data issues, such as siloes or lack of context. Further, more than half reported that legacy data systems are preventing them from scaling agentic AI and are having a “significant negative impact” on their AI ROI. High latency has also hampered AI from making decisions at “high velocity.”

“To make good decisions and take effective action, agentic systems need a data foundation that is multimodal, context-aware, and instantly available,” the report noted. “Legacy data systems struggle to meet these demands, ultimately compromising AI trustworthiness.”

Google and MIT identified several reasons that enterprises struggle to deploy AI agents, most notably:

• Entrenched siloes: Data sits in disconnected systems, or is “pocketed away” in different departments with no integration layer; it could also be in outdated formats, old management platforms or logs, or on obsolete IoT devices.
• Difficult-to-access data: “Dark” or unstructured data is contained in different formats like images, video, or PDFs.
• Insufficient access to real-time data: Legacy batch processing architectures can make in-time action a challenge.
• Lack of context: Agents often receive basic metadata rather than enterprise-specific semantics, so they struggle to make relevant connections or suggestions.

“Without this deep understanding, data cannot be highly relevant to specific use cases,” the report noted.

‘Data leaders’ versus ‘data laggards’

This is not to say that enterprises aren’t deploying AI; quite the contrary. Nearly all respondents (98%) to the Google/MIT survey are already using agentic AI or plan to soon. One in 10 is using it widely and nearly three quarters have deployed it in a limited fashion.

The most common uses for AI agents right now are in customer service (routing requests and resolving issues), IT systems management (managing user access and incident response), and IT security (anomaly detection and threat scanning). In the near future, the survey said, enterprises also plan to use AI agents in HR, finance, and supply chains.

“It is easy to understand why companies are eager to put AI agents to work,” the report noted. Agents can supercharge productivity and efficiency so employees can turn to more strategic work.

The enterprises seeing the most success are what Google and MIT refer to as “data leaders,” those that give AI systems access to more than 70% of their data. “Data laggards,” by contrast, share just 30% or less of their data with AI. Interestingly, 100% of data leaders say their agents make “mostly” or “consistently” accurate and relevant decisions, while just 22% of data laggards say they have that trust.

Agents need what the report calls “frictionless access” to operating systems, multimodal data, and context, helping them understand how data maps to different teams’ goals. “The most important initiative to enable scaling among all respondents is improving access to structured and unstructured data for AI agents.”

To successfully scale agents, the report recommended that enterprise leaders prioritize several data initiatives. First, improve access to data; discover, inventory, and classify data, both structured and “dark”/unstructured. After governance principles are applied, information can then be extracted and connected to AI agents.

Next, “put a premium on context” by giving agents enterprise-specific data. Replace batch processing with streaming and event-driven pipelines as well.

Finally, think AI-native. “AI-native systems are designed to operate in an AI environment and built from the outset to leverage AI for data management and decision-making,” the report noted.  

AI needs multimodal cloud environments

AI needs a lot of data that is often spread across on-premises systems and cloud, SaaS, and edge environments. In fact, 97% of respondents to Cloudera’s survey said they move data between environments monthly, and nearly one-third do so daily.

However, nearly three-quarters (73%) said AI integration makes data governance more complex.

“Enterprises had good governance systems when humans were the only ones accessing data manually,” the report noted. “But when thousands of agents provide support to employees, customers, or partners the story changes and governance requires another dimension.”

This makes private AI and data sovereignty critical, Cloudera said. Enterprises should have a data foundation that is unified and provides control over 100% of organizational data, wherever it resides. They must also think about where AI workloads run, while still maintaining control over sensitive data and keeping cost controls and flexibility in mind.

One notable trend Cloudera uncovered is a “resurgence” of on-premises and private cloud environments. Over the last 12 months, 66% of respondents moved AI workloads from public cloud back to on-premises or private cloud environments. And 25% said they plan to place more emphasis on a hybrid-first approach, 24% said they are planning to increase on-premises spend, and 22% plan to increase edge spend.

“IT leaders are putting more emphasis on moving workloads to the environments that they are best suited for, based on performance, cost, latency, governance, availability, and accessibility,” the report stated.

On the modern factory floor, the phrase “industrial equipment” no longer tells the whole story. It conjures images of steel, hydraulics, conveyor belts and machinery built to perform the same task with unwavering precision day after day. Physical engineering remains fundamental, of course, but it’s no longer the sole measure of a machine’s value. The next generation of machines have capabilities that depend on far more than the factory floor, continuously exchanging information with cloud platforms, data centers and AI systems that allow them to act autonomously and “self-improve” long after they’ve been deployed. A robotic arm isn’t simply running a predefined script anymore – it’s generating a constant stream of operational intelligence that reveals how it is performing, when and whether it needs attention, and how production can independently improve itself and become faster, safer and more efficient tomorrow than it is today.

This new functionality is redrawing the concept of ownership for manufacturers. Increasingly, the asset is not just the machine itself, but the flow of data that supports it and reveals clues about its functionality. Every production cycle enriches digital models, refines predictive algorithms and deepens operational understanding, turning what was once a static piece of equipment into something that continuously improves over time. The term “phygital” has emerged to describe this convergence of physical infrastructure and digital intelligence, but whatever terminology ultimately sticks, the outcome will be the same. As manufacturing enters an era where competitive advantage is increasingly shaped by software, analytics and real-time AI inference, CIOs are having to think very carefully not just about who owns the machine on the factory floor, but who controls the data that turns that machine from “dumb iron” into something that can “think” intelligently.

Manufacturing has entered its software-defined era

The physical engineering on display on factory floors is already impressive. Autonomous haul trucks can navigate vast mining sites without drivers, robotic arms can self-adjust their movements in relation to contextual cues, and in the case of so-called “dark factories,” entire production lines can operate 24/7 for a long time without a single person on the factory floor. Every movement, vibration, temperature change and production cycle becomes part of a data-driven feedback loop that allows software to refine performance, anticipate failures and adapt operations contextually in ways that simply weren’t possible when industrial equipment functioned in siloes.

According to Deloitte’s 2025 Smart Manufacturing and Operations Survey, 92% of manufacturers believe smart manufacturing will be the primary driver of competitiveness over the next three years, while 78% are allocating more than a fifth of their improvement budgets to smart manufacturing initiatives. Those figures bring home the fact that industrial performance is no longer determined solely by what happens inside a machine, but by how effectively the data ecosystem it lives in functions as a whole.

Every smart factory runs on an invisible supply chain

Every intelligent machine exists within a much broader ecosystem that stretches far beyond the walls of a factory, connecting equipment manufacturers, cloud platforms, systems integrators, AI providers and operational teams through a constant flow of data. It’s easy to think of a production line as a collection of individual assets working side by side, but the reality is far more interconnected. Each machine is both producing and consuming information throughout the working day, allowing decisions made in one environment to influence outcomes somewhere else. A software update developed by an equipment manufacturer, for example, might be informed by performance data gathered from thousands of identical machines operating around the world, with improvements delivered back to the factory almost as quickly as they’re identified.

From that perspective, data begins to resemble a supply chain in its own right. Manufacturers have spent decades refining the movement of raw materials because every unnecessary delay carries a measurable operational cost, and that same principle now applies to information. The data flowing from production equipment, the analytics returning from cloud platforms, and the insights generated by AI have become just as vulnerable to delay as the components arriving at the loading dock.  According to the International Federation of Robotics, more than 4.8 million industrial robots are now operating in factories worldwide, and each one contributes to a growing stream of operational data that has become inseparable from the manufacturing process itself. The challenge for CIOs used to be, “How do we connect these environments?”, but now it’s “How do we ensure the data moving between them arrives with the speed, visibility and control needed to keep pace with modern manufacturing?”

The importance of network architecture

The value of data used to be measured solely by its accuracy, but now it depends on how reliably it can move between the organizations that create it, analyze it and act upon it. A predictive maintenance platform can’t identify an emerging fault if telemetry arrives too late, just like a digital twin is only as useful as the information it receives. As we bridge from Industry 4.0 to Industry 5.0, the network itself is becoming an active participant in the production process, prompting CIOs to think differently about connectivity.

Modern manufacturing depends on a growing ecosystem that needs to exchange data in near real time. Rather than relying on unpredictable routes across the public Internet, many organizations are turning to direct interconnection in the form of internet, cloud and AI exchanges, which act as neutral meeting points where enterprises and their suppliers, as well as network operators, cloud providers and digital or AI service providers, can establish direct, private connections with one another. By shortening the path data has to travel and avoiding unnecessary “hops” and congestion, these platforms reduce latency, improve resilience and give organizations far greater visibility and control over how production-critical information moves.

Every revolution in manufacturing has been defined by the emergence of a resource that reshaped how value was created, whether that was steam, electricity or silicon. Industry 5.0 is introducing another, albeit one that can’t be stored in a warehouse or delivered on a truck. Data has become the factory’s most valuable raw material, and controlling its movement is every bit as important as controlling the movement of physical goods. The term “industrial equipment” may continue to describe what’s happening on the factory floor, but it no longer captures where competitive advantage is really being created. Increasingly, the intelligence surrounding a machine is becoming just as valuable as the machine itself, and the networks carrying that intelligence are becoming part of the production process in their own right.

Earlier this year, the National Bureau of Economic Research released survey results from over 6,000 U.S. leaders showing that while AI adoption is widespread at 69%, we’re seeing little to no impact on productivity. Anecdotally, we’ve seen leaders from top companies echo that refrain.

It’s the reckoning many CIOs, CTOs and COOs are navigating as we enter the last half of the year. The most humbling part is knowing it’s a management problem we created by treating AI like it was exempt from the rules we apply to every other enterprise tool.

Part of this has to do with how AI entered the market. The tools that sparked its mainstream adoption arrived as consumer products before enterprises had governance frameworks to absorb them. Enterprises were left playing catch-up as they grappled with IP and data security concerns, inadvertently fueling shadow AI as employees leveraged these tools to get ahead and eventually, keep pace, at work. What this created was a sense of entitlement that is challenging to unravel.

Like the internet writ large, employees have grown to expect unlimited access, and organizations played along. But this idea warrants a pause. When did we last roll out Salesforce to everyone who asked without a use case? AI got a pass because it felt different. In truth, it isn’t. It’s another tool that enterprises need to manage.

Three levels every information and technology leader has to solve

It’s helpful to look at this as a three-level evolution framework.

Level one is adoption — are people actually using it well? Level two is budget control — what are we spending and on what? Level three is justification — can we demonstrate the return?

Most companies are still at level one. Deloitte reported in their 2026 State of AI in the Enterprise report that only 25% of respondents have moved 40% or more of their AI experiments into production to date. The minority that have moved pilots to production are grappling with the budget and trying to figure out how to justify the costs and quantify the gains.

The problem is, you can’t prove what you didn’t have to hire because of AI. There is no parallel universe where you can walk into the CEO’s office and say I need five more people in finance, but in this universe, with AI, I didn’t.

The organizations that wait for a clean ROI model before making any decisions will spend themselves into the trough of disillusionment before they find one. The smarter move is to start treating it as a discipline you build.

What managing AI like a tool actually looks like

To some, governance sounds like restriction. But the discipline is more about matching the right tool to the right use case, and making the sanctioned path easier than the workaround.

Take shadow AI. The instinct is to lock things down. But when employees start building internal apps with company data and hosting them on free public platforms, the answer isn’t another policy. By the time the policy is written, the data is already public. Instead, you need to build an internal alternative that does the same thing without the exposure. Give people a path. If you don’t, they build their own, and you won’t know about it until something goes wrong.

The same principle holds for conflicting data. Two departments pulling AI-generated recommendations from the same underlying data and arriving at different conclusions isn’t an AI problem. It’s a data and definitions problem. AI just made it impossible to ignore. Say marketing claims they brought $50 million in the pipeline, and sales claim they brought $50 million as well.But the company actually has $75 million in pipeline. Someone is counting the same deals twice under different definitions. The CIO’s job is to enforce one source of truth. If your dashboard doesn’t match the authoritative one, your dashboard is wrong. That’s the only way the organization can function.

And it applies to cost, too. Not every workflow needs the most expensive model. Not every employee needs full AI access. If someone is using a top-tier model to summarize email because nobody told them there was a cheaper option that does the job, that’s a gap that CIOs need to address. The CIO’s job is to build the layer that makes the right choice the obvious one, and to provide sanctioned alternatives so employees aren’t left building their own.

That’s what actually reduces shadow AI, conflicting data and runaway spend: Alternatives, visibility and a single source of truth.

The CIOs getting real value from AI right now aren’t the ones who said yes to everything. They’re the ones who asked the same questions they’d ask about any other enterprise investment: What does it do, who actually needs it and what are we getting back?

AI is a remarkable tool. It’s also just a tool. It doesn’t exempt you from the management discipline you apply to every other system in your stack. We didn’t roll out Salesforce to everyone who asked without a use case. We shouldn’t have done it with AI either, and the organizations that did are now living with the consequences:  Six-figure token bills, shadow apps on public URLs, dashboards that contradict each other and a CEO asking what exactly he got for the investment.

The answer to that question is available. But only if you built the infrastructure to find it.

In “Micro and macro agents: The emerging architecture of the agentic enterprise,” I proposed a three-layer architecture for enterprise AI.

1. Micro agents execute specialized tasks.
2. Macro agents orchestrate end-to-end business processes.
3. Meta agents provide governance through monitoring, compliance, security, and human oversight.

As enterprises begin deploying thousands — and eventually tens of thousands — of autonomous agents, token costs have become a major concern. According to Gartner, rising token-driven AI spend is straining budgets and challenging cost justification.

To track this economic concern, meta agents should do more than simply being the governance agents.

They should become the economic intelligence layer of the enterprise.

Their responsibility is not only ensuring AI behaves responsibly.

It is ensuring AI creates measurable business value.

The missing economic model for AI

Every major technology revolution eventually develops its own economic framework:

• Manufacturing measured productivity.
• Cloud computing measured infrastructure utilization.
• Digital businesses measured customer acquisition costs and lifetime value.

The agentic enterprise now requires its own financial discipline. Every AI prompt. Every reasoning cycle. Every interaction between agents. Every autonomous workflow.

Tokens have quietly become the operational currency of enterprise AI. Tokenomics is now a foundational part of enterprise AI architecture.

Yet today, most organizations measure only one thing: Cost. How many tokens were consumed? Which models cost the most? What was the monthly inference bill?

These are useful operational metrics.

They are not strategic business metrics. Boards rarely ask how much electricity a factory consumed. They ask how much value the factory produced.

Enterprise AI deserves the same conversation.

This is where I was thinking about the laws of physics.  Based on physics laws,  energy cannot be created or destroyed. It is transformed into another form. Electricity becomes light. Chemical energy becomes motion. Solar energy becomes electricity.

Enterprise AI offers a similar management lesson.

Intelligence must be transformed into value

Tokens are not valuable because they are consumed. They become valuable only when they are transformed into business outcomes. A faster loan application decision. A fraud detection. A better customer experience. Higher software quality. Greater employee productivity. A new business opportunity.

This leads to what I call return on tokens (ROT).

ROT measures how effectively an organization converts token consumption into measurable business value.

Instead of asking, “How many tokens did we consume,” leaders should ask, “How much enterprise value did every million tokens create?”

The Second Law of Thermodynamics tells us something equally important: Every energy transformation introduces inefficiencies. Although total energy is conserved, some inevitably becomes less useful for doing work.

Enterprise AI behaves similarly.

The second law: Every AI transformation creates friction

Not every token creates value. Some tokens are spent on repeated reasoning. Some generate redundant conversations between agents. Some support oversized context windows. Some produce hallucinations requiring correction. Some route simple tasks to unnecessarily expensive models.

The tokens are not lost. But they create very little useful business work.

I refer to this as token entropy. Token entropy represents the portion of AI activity that consumes intelligence without producing proportional business outcomes.

Every agentic enterprise will experience token entropy. The organizations that win will be the ones that continuously identify and reduce it.

Beyond energy: The importance of exergy

Thermodynamics offers another concept that is even more relevant. It is called Exergy.

Unlike energy, exergy measures the amount of energy that can actually be converted into useful work. Two systems may contain the same amount of energy while producing dramatically different levels of useful output.

The same principle applies to enterprise AI. Two organizations may consume exactly the same number of tokens.

One generates meeting summaries.

The other transforms loan  processing, accelerates software development, detects fraud, improves customer retention, and creates new revenue streams.

Their token consumption is identical. Their business impact is not.

Borrowing it as a management analogy, not claiming that AI tokens literally obey the thermodynamic definition of exergy. I think of this as token exergy. It’s not that AI tokens literally obey the thermodynamic definition of exergy. 

Token exergy measures how much of an organization’s AI intelligence is converted into useful business work. It is not enough to consume tokens efficiently. Organizations must convert those tokens into outcomes that matter.

The meta agent evolves

This is where meta agents become transformational.

Today we think of them as governance agents. Tomorrow they become economic governors.

Meta agents continuously monitor every interaction across the enterprise and answer questions such as:

• Which agents produce the highest ROT?
• Where is token entropy increasing?
• Which workflows generate the highest token exergy?
• Which models deliver the greatest business value per token?
• Which agents should use smaller models?
• Which prompts should be optimized?
• Which workflows require human intervention?
• Which autonomous processes should be redesigned?

Meta agents no longer simply supervise AI. They optimize its economics.

The economic intelligence layer

The architecture now becomes complete.

• Micro agents: Perform work.
• Macro agents: Coordinate work.
• Meta agents: OGovern, observe, optimize, and continuously improve the economics of intelligence.

Their objective is straightforward:

• Maximize return on tokens.
• Minimize token entropy.
• Increase token exergy.

This represents a shift from AI governance to AI economics.

The executive dashboard of tomorrow

The executive dashboard of the future will not focus solely on infrastructure metrics. It will measure intelligence performance.

Imagine a boardroom dashboard displaying:

• Return on tokens (ROT)
• Token entropy index
• Token exergy score
• Business value per million tokens
• Agent productivity index
• Cost per autonomous decision
• AI value by business unit
• Human escalation rate
• Model effectiveness score

These metrics move AI discussions beyond engineering. They make AI accountable for business outcomes.

A new responsibility for CIOs

The next generation of CIOs will not simply deploy AI. They will manage an economy of intelligence.

Their role will resemble that of a portfolio manager — allocating AI capacity where it creates the greatest enterprise value, reducing waste, and continuously improving the productivity of every autonomous workflow.

That responsibility cannot be fulfilled by dashboards alone. It requires an intelligent layer capable of observing, learning, and optimizing the entire agent ecosystem.

That is the emerging role of the meta agent.

The next competitive advantage

Every technological revolution rewards organizations that learn to measure what others overlook.

Factories measured productivity — not fuel consumption.

Digital businesses measured customer engagement — not server utilization.

The agentic enterprise will reward organizations that measure intelligence itself.

The winners will not be those deploying the largest models. Nor the most agents. Nor consuming the fewest tokens.

They will be the organizations that continuously maximize return on tokens, relentlessly reduce token entropy, and increase token exergy.

I believe this is the next evolution of the agentic enterprise.

Not simply governed intelligence, but economically optimized intelligence.

The AI adoption spending spree is over. Time to focus on value.

And in that future, meta agents will serve not only as the guardians of AI — but as the stewards of enterprise intelligence economics.

Through this framework I strongly believe that executives can easily remember the key measures for economic intelligence. 

• ROT (return on tokens): How much value did AI create?
• Token entropy: Where are we wasting AI intelligence?
• Token exergy: How effectively are we converting AI intelligence into useful business work?

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Enterprises have spent decades learning how to audit people and software. Agentic AI creates a third category: systems that interpret instructions, call tools and act across workflows without a mature assurance model built around them.

In my work as a leader and investor across technology-enabled businesses, I have spent years around automation, cybersecurity, compliance, workflow design and board reporting. I have watched management teams gain confidence from dashboards, policies and approval records, then face a harder question when a board member, auditor or regulator asks whether the controls performed as intended.

Agentic AI complicates that question because a single outcome may pass through several systems. An agent can collect information, choose a tool, produce code, route a request and hand work to another agent before a person approves the result. No single manager may have observed the full path.

Executive accountability remains human even when the operating activity becomes more autonomous. The CIO may have to explain who authorized the activity, whether the agent stayed within its approved purpose and what evidence supports management’s answer.

In a recent framework for frontier AI, Google DeepMind CEO Demis Hassabis proposed an independent standards body that could evaluate advanced models before deployment and address critical vulnerabilities after release. His proposal focuses on frontier models, but the principle carries into the enterprise: expanding autonomy creates a corresponding need for independent assessment.

My rule for this stage of adoption is straightforward: no agent should gain more autonomy than the company can verify.

The enterprise audit model was built for people and software

Companies have spent decades building controls around people. Employees have job descriptions, reporting lines, approval limits and access rights. When someone leaves, an established process removes access and transfers responsibility.

Traditional software also fits a familiar structure. A program follows defined instructions inside systems with owners, release procedures, test records and change controls. Complexity can make review difficult, but the accountability chain is usually visible.

An AI agent sits between those categories. It operates through software while interpreting instructions with room to choose a path. Its behavior may change when the model, prompt, connected data, available tools or surrounding workflow changes. A control approved during deployment can weaken months later without an obvious change to the application.

Standards are still developing as adoption accelerates. In February 2026, NIST launched an ⁠AI Agent Standards Initiative focused on secure operation and interoperability for agents capable of autonomous action.

Par Chadh

Gartner predicted that 40% of enterprise applications would include task-specific agents by the end of 2026, up from less than 5% in 2025. More than 40% of agentic AI projects could be canceled by the end of 2027 because of escalating cost, unclear business value or inadequate risk controls.

IBM’s 2025 Cost of a Data Breach research found that ⁠13% of surveyed organizations reported breaches involving AI models or applications. Among that group, 97% reported inadequate AI access controls. 63% of organizations in their study lacked governance policies for managing AI or preventing shadow AI.

Par Chadh

One team may approve an agent, another may connect it to data and a third may own the workflow. Management still carries responsibility when the agent exposes information, produces an error or acts outside its approved purpose.

Two-thirds of CIOs and CTOs surveyed were being held ⁠accountable for AI systems they did not fully control. Seventy percent said technology was spreading across the business faster than IT could track it, while 77% said adoption was outpacing governance capabilities.

Par Chadh

The survey findings expose the boardroom gap: management carries accountability while control remains distributed across teams, systems and workflows. An assurance model must provide more than a statement of intent.

Assurance must prove where the boundary held

Policies, dashboards and logs help establish control. Assurance begins when the company tests whether an agent stayed inside the conditions management approved.

Before deployment, management should document the agent’s business purpose, accountable owner, systems touched, allowed actions and stop conditions. Testing should then determine whether the agent can reach information outside its scope, call an unapproved tool, continue after a stop condition or carry an incorrect assumption into another system.

I would treat an agent’s autonomy as a renewable license because its operating condition will change after launch. Renewal should follow any material change to the model, connected data, available tools, workflow or authority. NIST’s work on the challenges of monitoring deployed AI systems identifies drift, fragmented logging and immature standards as barriers to post-deployment oversight, while the World Economic Forum recommends scaling safeguards with an agent’s autonomy, authority and complexity. A change-triggered review ties assurance to the version of the agent and workflow in use, giving the CIO a defensible basis for continued authority.

Consider a coding agent that begins by drafting test cases, then gains access to repositories, tickets, CI/CD tools and production documentation. A production change could involve an instruction, code, a tool call, an automated test, a ticket update and human approval. Assurance must show how the result was produced, which systems participated, whether the agent crossed a boundary and how exceptions were handled.

For every agent with meaningful operating authority, I would expect four connected records: the approved baseline, boundary-test results, a history of behavioral drift and an account of exceptions and interventions. Together, they give management a record that can support a board discussion, audit or regulatory response without depending on the technical team’s memory.

Autonomy should scale only as fast as assurance

Internal teams will remain responsible for designing controls and operating the environment. At board-level scale, management also needs review independent from the people who built and run the agent.

I have seen management ask auditors or other independent certified professionals to sign off on a system. They cannot sign when they have not completed the work required to support that opinion. Leadership wants confidence, the board wants an answer and the independent party needs a body of evidence that can be tested.

Agentic AI will make that evidence harder to assemble after an incident. Records have to be produced while the work occurs. Once an agent has acted across several systems, reconstruction may depend on logs created by different vendors, teams and tools. Missing context can turn a clear technical event into an uncertain management explanation.

CIOs should design assurance into the workflow. The operating record needs to capture approved purpose, tested boundaries, material changes, exceptions, human interventions and unresolved findings. Independent review can then examine whether the control operated during the period management is being asked to discuss.

The operating record gives executives a basis for changing an agent’s authority. Broader responsibility should follow tested boundaries and a clean exception history. Drift or repeated intervention should pause expansion until the cause is understood.

Before approving broader use, I would ask:

• What authority has the company granted?
• Which tests show the boundary holds?
• What record will remain available months later?
• Who carries accountability when the record shows a failure?

Those questions create a management standard for deciding whether an agent is ready to move from a limited workflow into broader enterprise operations.

Enterprises have spent decades learning how to audit people and software. Agentic AI creates a third category that requires its own assurance model. The next discipline for the enterprise is auditing autonomy through approved behavior, tested boundaries, monitored change and documented intervention.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

Ask most enterprise technology teams where they spend their AI strategy energy and you will get the same answer: figuring out which model to use. It feels like the right question. As organizations move from pilots into production and the real compliance, cost and continuity risks appear, it turns out to be the wrong one.

Writing on CIO.com this year, Floyd DCosta argued the divide is between enterprises that own their AI and those that rent it, and later that closed-model dependency is outsourced intelligence with a vendor kill switch in your operations. He is right. But the ownership question raises a harder one: own it how? I believe the answer is open-weight and open-source models, not because they are cheaper, but because they are structurally better suited to how serious organizations need to govern and protect AI at scale.

Why closed models create governance problems

Building an enterprise AI program on closed, proprietary models from a single external provider is not a technology decision. It is a governance liability. The data confirms the exposure is already real.

A June 2026 IBM Institute for Business Value study of 1,000 senior executives found that 91% do not fully understand their AI vendor dependencies, 71% said switching providers would be difficult, and 81% said a seven-day vendor outage would cause severe disruption. These figures describe the baseline condition of enterprise AI in 2026.

Think about what you give up. You cannot audit the training data. You have no visibility into how the model changes between versions. Your cost structure is set by someone else’s pricing team. And if that provider faces a government directive, a supply disruption or a commercial decision to reprice, you have no leverage and often no warning. For a generic SaaS tool, that is an inconvenience. For organizations in defense, healthcare or financial services, where data handling is regulated by law and audit trails are mandatory, a vendor changing model access terms overnight is a compliance event.

The most vivid demonstration came in June 2026, when the U.S. Commerce Department ordered Anthropic to suspend access to Fable 5 and Mythos 5 for all foreign nationals, including its own non-citizen employees. The result was a hard global shutoff for every customer, with no advance notice. Enterprises with production workflows on those models were left with nothing. The precedent is now set.

This is what AI governance exposure looks like in practice. When your intelligence layer sits entirely outside your control, a single government directive, pricing change or vendor decision can bring your AI operations to a halt. Having seen organizations scramble through exactly this scenario, I can say the ones with no continuity plan are the most exposed. The IBM numbers confirm it: most enterprises have not built the visibility, let alone the architecture, to absorb this kind of disruption. The question is not whether it will happen again. It is whether your architecture is ready when it does.

Open-weight models have changed the equation

Until recently, the argument for closed frontier models was simple: they were dramatically better. That gap has narrowed faster than most enterprise technology leaders anticipated, and the conversation has shifted from capability to control.

Open-weight models, including Meta’s Llama family, Alibaba’s Qwen series, Zhipu AI’s GLM and DeepSeek, have moved well past the research stage. They are running in production at serious organizations, and not because those organizations could not afford anything better. They chose them because open-weight models give them something closed models cannot: control.

Airbnb’s adoption of Alibaba’s Qwen makes the case plainly. CEO Brian Chesky stated that the company relies heavily on Qwen to power its customer service agent, describing it as “very good” and “fast and cheap,” while noting that OpenAI’s SDK was not ready for the depth of integration Airbnb needed. The agent runs across 13 different models. That is not a cost-cutting move. It is a deliberate multi-model architecture built around control, not just capability.

Microsoft’s evaluation of DeepSeek V4 for Copilot Cowork, reported by Axios in June 2026, tells the same story. The company is exploring a self-hosted DeepSeek to replace the Anthropic and OpenAI models powering its enterprise agentic product, driven by unsustainable costs at scale. Charles Lamanna, Microsoft’s executive vice president for Copilot, agents and platform, told Axios: “We have users who do hundreds of tasks a week… the consequence is the costs can go very high.” The IBM study’s full findings add context: organizations pay 2.8 times more in token processing when AI runs far from the data it depends on.

When Airbnb and Microsoft are making these choices in production, the market signal is unambiguous. Open weight is not a fallback. It is the architecture direction serious enterprises are moving toward.

Some of these models are Chinese in origin, and yes, that has drawn attention from U.S. lawmakers. Those are real conversations worth having. But here is the practical point: a model running inside your own infrastructure, under your own security controls, gives you more governance than a closed model running on a server you do not own, regardless of where either was built.

Three forces accelerating the shift

Three things are pushing enterprises in this direction, and none of them are going away.

• Cost at scale. Every API call compounds. Open-weight models on your own infrastructure convert that variable cost into one your organization controls. The IBM study found misaligned AI infrastructure costs enterprises 2.8 times more in token processing. Microsoft’s DeepSeek evaluation is that logic at the largest scale in the industry.
• Data protection. Consider what this means for a hospital routing patient records through a third-party AI API, a defense contractor using a closed model to analyze procurement data, or bank feeding client financials into an external inference endpoint. In each case, compliance is contingent on a vendor relationship the organization does not fully control. HIPAA does not care whether your AI vendor had a good SLA. FedRAMP authorization does not transfer because a model performed well on a benchmark. Regulators expect organizations to demonstrate control over where sensitive data goes, and a closed frontier API is not a defensible answer. The IBM study found 68% of executives say meeting data residency and sovereignty requirements across geographies is already challenging. Open-weight models deployed within your own environment remove that exposure entirely. The data does not leave your perimeter, full stop.
• Sovereignty. The Fable 5 episode made clear that your AI capability is only as sovereign as the provider you depend on. For defense primes, intelligence contractors and any organization operating under jurisdiction-specific regulations, this is an existential design question, not a preference. Open-weight models, deployable under your own governance in any environment, are the only architecture that resolves it.

What this means for enterprise AI architecture

In conversations with technology leaders, the same pattern keeps surfacing: organizations that started with a single frontier provider for speed are now the most constrained when they try to scale, govern or adapt. This is especially acute in regulated sectors. A healthcare organization that built clinical documentation on a closed frontier model faces a hard question every time that vendor changes its data processing terms. A defense contractor with a closed model embedded in its logistics pipeline must revisit its authorization to operate every time the model updates silently. The implication is not to abandon frontier models entirely. It is to stop building AI programs that depend on them as the sole or default layer.

The practical answer is a multi-model architecture: frontier models where the capability genuinely justifies the cost and the data exposure, open-weight models running on your own infrastructure for everything else. Not every task needs the most powerful model available. And not every task should leave your perimeter.

DCosta framed the coming divide as between AI owners and AI renters. I would take that one step further. The organizations that will genuinely own their AI are the ones building the infrastructure, governance and internal capability to run open-weight models on their own terms, right now. The ones that continue to depend entirely on closed frontier providers are not owners, whatever they call themselves. They are renters, and their leases can be terminated, repriced or restricted at any time. The IBM study makes the stakes clear: 57% of executives say replacing a core AI model would require significant decoupling or a full rebuild. The longer you wait to build portability in, the harder it gets.

The best AI model is not the one with the highest benchmark score. It is the one that fits into an architecture your organization governs.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

As enterprises grapple with governing growing fleets of AI agents, Snowflake has unveiled Cortex AI Gateway, a new runtime control plane designed to help organizations track agent actions, enforce policies, and manage AI spending across models and tools.

Built on technology acquired through the purchase of Natoma earlier in May, Cortex AI Gateway “acts as an execution layer” that applies governance policies defined in Horizon Catalog while routing requests and enforcing cost controls across first- and third-party AI agents in an enterprise, said Artin Avanes, head of core data platform at Snowflake.

“Its role is to provide a trusted control plane that governs how AI agents securely access models, tools, MCP servers, enterprise systems, and data across increasingly diverse AI environments as enterprises adopt more models and agents for their applications and workflows,” Avanes added.

That control plane, according to analysts, could help enterprises address “a real gap” as they scale agentic deployments.

“Most enterprises cannot see or govern agent activity consistently across models, tools, MCP servers, and enterprise systems, as most AI gateways just route models and log prompts,” said Michael Leone, principal analyst, Moor Insights and Strategy.

“Enterprises need to know which agent acted, who authorized it, what resources it used, and what happened at each step. Without that runtime evidence, firms cannot reliably secure, audit, or contain agentic workflows,” echoed Stephanie Walter, practice lead of AI stack at HyperFRAME Research.

The new offering, according to Leone, should also help CIOs reduce the burden of governance operations as agents scale.

The key is embedding AI agent governance, including related FinOps, where enterprise data is already governed, instead of creating another management layer, which is what Cortex AI Gateway does, Leone said.

That attempt to combine FinOps and governance within one control plane is likely to be seen as a “logical” move by CIOs, as each of these functions requires “much of the same” shared data and traceability logs to accomplish their respective goals, said Scott Bickley, advisory fellow at Info-Tech Research Group.

More so because model and agent providers have been shifting toward consumption-based pricing, making it harder for CIOs to understand who is using AI resources, how much they cost, and whether spending remains within policy, Bickley pointed out.

For developers, though, the new offering can cut both ways.

If implemented well, the Gateway should simplify development by simplifying tedious tasks and consolidating separate provider credentials, provisioning of logging frameworks, cost and usage instrumentation, and access control mechanisms with one application layer, Bickley said.

Otherwise, it could become a bottleneck with developers being forced to use multiple gateway products, resulting in development delays, Bickley added.

The offering is slated to enter public preview soon.