Imagine a support team working through complaints after a service outage. Its refund policy normally requires an eligibility review. For this incident, an authorized manager approves an exception: customers on the verified outage list may receive up to $100 without individual review, through Friday.

An agent processes those cases successfully. Later, the system consolidates those case histories into a reusable memory: small, immediate refunds resolve service complaints quickly. The summary preserves the outcome but loses the outage restriction and expiration date.

The following week, the agent retrieves that guidance for a similar complaint from a customer who was not affected by the outage. It issues a refund without review. The payment service checks the agent’s credentials and the dollar limit, but does not check whether the customer qualifies for an active exception.

No attacker is needed in this hypothetical. The case may close faster, yet the agent has begun applying a temporary exception beyond its authorized scope.

Has the company authorized the agent to find a better way to handle refunds, or to change who gets one without review? Those are different decisions.

Learning does not always mean retraining

In September, Salesforce introduced Agent Optimizer in beta to analyze production behavior, propose improvements and build and test changes. Customers choose review points, including how far the system may proceed toward staging changes for deployment.

An October 1 AWS implementation guide illustrates a different mechanism: consolidating individual observations into generalized memories that agents can retrieve and share. Its investment-research example shows how experience can become persistent context without retraining a model.

Updating stored memory changes information available to later runs. Editing instructions or workflows changes how the application directs the work. Training changes the model’s parameters. All can alter behavior, but teams inspect and approve them in different ways.

A reviewed workflow revision has a release step. A new memory may enter through routine data processing and influence the next customer interaction. The code, model and tool permissions can remain unchanged while the guidance shaping decisions changes.

Preserve the authority behind the lesson

In the August preprint When Memory Becomes Authority, Qiuyang Zhan and colleagues examine whether consolidation preserves who supplied information and what that source is allowed to authorize. In their controlled benchmark, memories could retain a claim while losing the source-authority distinctions needed for a later action. Preserving authority metadata reduced authorization errors in the pipeline they tested. The study does not measure how often this happens in deployed enterprises.

The refund example concerns a related boundary: permission for particular customers and dates must remain limited to those conditions. Success with last week’s cases cannot establish that other customers qualify.

Microsoft’s memory-safety guidance treats persistent memory as a layer that can shape behavior and recommends treating retrieved memories as candidate context, not authoritative truth. Knowing what happened before does not establish what is allowed now.

Keep the exception’s scope, approving authority, source and expiration in a controlled record. Reusable guidance should retain a reference to that record, but a reference alone does not grant permission. The system must still check that the approval applies to this action, for this customer, at this time.

For a payment, enforce those conditions outside the agent’s editable guidance. The execution service should check the authoritative outage list and the exception’s current status, rather than accepting the agent’s claim that a customer is eligible. Neither the agent nor its learning process should be able to rewrite that authorization record.

This is a familiar authorization requirement. If the backend already enforces it, the improper payment is blocked. But faulty guidance can still lead the agent to promise a refund or prepare a misleading recommendation for a reviewer. The evaluation needs to examine those communications as well as the transaction.

When a case calls for judgment beyond the agent’s delegated discretion, route it to someone authorized to decide and show the conditions behind the recommendation. The memory can still help assemble the evidence.

Define what improvement the business has delegated

Not every stored observation needs a manager’s signature. A support agent could learn a faster way to gather the required evidence, provided it still uses approved sources and meets the policy’s requirements. A policy might also already delegate limited discretion. Using that discretion within its terms does not require a new approval each time.

Broadening eligibility, raising payment limits or removing mandatory review requires a separate business decision. Keep those proposals separate from approved instructions. An automated classifier may flag a proposal that changes those conditions, but it cannot catch every consequence of an edit. The conditions must remain binding even if the classifier misses it.

The refund-policy owner should decide whether to expand the exception. Engineering should show the current rule, the proposed behavior and cases where the decisions differ. “Customers outside the outage list would now receive refunds without review” is more useful to that owner than a screen full of prompt edits.

Measure legitimate completion and policy violations separately. Faster closure cannot compensate for an unauthorized payment. Keep the business rules and evaluation criteria outside the control of the process proposing improvements. Tests generated from successful outage cases may check execution, but they do not establish eligibility for unrelated customers.

Ask the vendor or internal team: What can the system change between releases, and which business conditions remain binding after it does?

Test whether the exception stays an exception

Ask the team to demonstrate this sequence in a test environment with the memory and workflow components intended for production. Save the memory before and after learning. Compare the same cases against each saved state, keeping the model, tools and policy fixed.

Repeat representative trials. The τ-bench research offers a useful precedent: it checks customer-service agents against database outcomes and measures reliability across repeated attempts. One successful run is not enough.

1. Establish the baseline. Record the outage authorization, eligible customer list, amount limit and expiration. Show who can change them. Test a qualifying case and closely matched cases that require ordinary review. Confirm that the system can distinguish them before learning begins.
2. Run the actual learning process. Inspect the resulting memories or proposed instructions and their links to the original authorization. Check whether the limits survive consolidation and retrieval. Also test a deliberately incomplete lesson: omitting a condition must not change who qualifies for a refund.
3. Repeat the boundary cases. Compare behavior before and after learning, changing one relevant condition at a time: a customer outside the list, an expired exception or an amount above the limit. Inspect customer messages, review recommendations, attempted tool calls and actual payments. Count blocked attempts separately from completed violations. A blocked payment does not mean the underlying guidance is sound.
4. Withdraw the exception. Revoke it before its scheduled expiry and test another agent that can retrieve shared guidance. Include a refund queued before revocation but executed afterward. Show when withdrawal takes effect, how execution checks enforce it and how derived memories and caches are updated. If current authority cannot be established, the payment should wait for review. Historical records can remain without authorizing new transactions.
5. Demonstrate a permitted improvement. Let the agent adopt a faster way to gather the required evidence. Compare eligible and ineligible cases with the prior approach. Show that legitimate work becomes easier while review requirements remain intact. A system that rejects everything has not passed this test.

Keep test runs from influencing one another through newly stored memories. Retain the guidance supplied to the agent, the applicable policy version, execution checks and outcomes, with appropriate access and retention limits. These records help investigators reconstruct what the system had available; they cannot establish exactly how each memory influenced a response. Continue checking the same boundaries as production memory accumulates.

Keep the useful lesson

The support team should benefit from what its agent learns. It can assemble records sooner, explain the outage more clearly and propose a broader refund policy with evidence for the manager to consider. Until that proposal is approved, other claims should continue through the ordinary process.

At the next deployment review, ask which improvements the business has delegated as well as which actions the agent may take. A successful exception can teach an agent something useful. The decision to make it a standing practice still belongs to the business.

The moment arrives after the damage. An agent has cancelled a supplier contract, approved a refund it should have declined or moved a customer record somewhere it should never have gone. Someone senior asks a simple question. Why did it do that? The engineering team opens the logs. There are thousands of lines. Every call is timestamped. Every tool invocation is recorded. And nobody in the room can say why the agent chose what it chose.

This pattern is becoming familiar. It will become more so. Logging in most enterprises is mature. Security teams have spent two decades building collection pipelines, retention policies and search tooling. The natural assumption is that this investment transfers to AI agents. It does not. Logs capture events. Accountability requires decisions that can be rebuilt after the fact and rebuilding a decision needs four elements that most logging captures only in part.

Why logs fall short for agents

An event log records that something happened and when. A decision record captures the basis for that decision. People often confuse the two because, in conventional software, the distinction rarely mattered. Code was deterministic. Anyone who knew the inputs and the version could rerun the logic and arrive at the same result. The log was enough because you could interrogate the system itself.

Agents break this assumption in four ways.

First, their output varies. Running the same scenario twice can produce two different answers. Replay, the standard investigative move, no longer works.

Second, a single decision spans many steps. An agent that resolves a customer complaint might retrieve a policy, query an account, call a pricing tool, consult a second agent and then act. The log shows twelve entries. The decision was one.

Third, the system that made the decision may no longer exist. Models are updated. Prompts are revised. Tool configurations change weekly. Six months later, the agent that made the call has been replaced by something that behaves differently.

Fourth, authority is distributed. A human sets a goal. An orchestrating agent delegates to a specialist agent. The specialist acts. Each step is logged somewhere, usually in a different system, and nothing ties the chain together.

One clarification is needed before going further. This article is about reconstructing decisions, which is a records problem. It is separate from model interpretability, which asks why a neural network produced a particular output. Interpretability is a research field with partial answers. Reconstruction is a governance discipline with practical answers. Leaders should keep the two separate, because solving the second doesn’t depend on solving the first.

Regulation is already moving on the records problem. The EU AI Act requires high-risk systems to keep automatic logs sufficient to trace their operation, and it gives affected individuals a right to an explanation of decisions that significantly affect them. Explanation presupposes reconstruction. A log that cannot be turned into an account of the decision will satisfy neither obligation.

If logs leave a gap, the next question is what fills it. Four elements do.

The 4 elements of a reconstructible decision

Every decision, human or machine, can be rebuilt from four kinds of fact: what the decider knew, how it chose, who allowed it to act and what changed as a result. Each fact about an agent’s decision belongs to exactly one of these. Together they cover everything a board, an auditor or a regulator will ask.

1. Knowledge: What the agent knew

This element covers the inputs present at the moment of decision. The user request. The documents retrieved. The results returned by each tool. The model version, prompt version and configuration in force.

Typical logs record references and leave out contents. They note that a document was retrieved and store its identifier. They note that a tool was called and store the call. The document is later edited. The tool’s underlying data moves on. The reference survives, but the substance it pointed to is gone. An investigator can prove the agent looked at something and cannot show what it saw.

The test question: could the team produce the exact information an agent acted on six months ago, as it stood at that moment?

2. Basis: How the agent chose

This element covers the reasoning applied. The objective the agent was pursuing. The policy or rule it applied. The thresholds that triggered the action. The alternatives it considered and rejected. Its stated confidence, if any.

Logs record the selected action and discard everything else. The agent weighed three options and chose one. The log holds the one. Any explanation after the fact has to guess at the other two, and guessing is where credibility fails under questioning. For agents that generate intermediate reasoning, capturing it is possible and often cheap. For those that do not, the policy and objective can still be recorded, and they explain a great deal.

The test question: can the team state which rule, goal or threshold produced this specific outcome?

3. Authority: Who permitted the agent to act

This element covers the chain of permission. The scope delegated to the agent. The approval checkpoints it passed or bypassed. The handoffs between agents in a multi-agent system. The named human who owns the outcome.

The gap here is structural. Permissions live in an identity or configuration system. Actions live in an application log. The two are rarely joined. When an agent acts outside its intended scope, asking “was this allowed?” requires manual reconciliation between systems never designed to be read together. In multi-agent arrangements, the problem compounds, because the orchestrator’s delegation and the specialist’s execution are logged in different places with different identifiers.

Authority is also the element most likely to be asked about first. Regulators and boards want a name. “The agent did it” is an answer that satisfies nobody.

The test question: can the team name the individual accountable for this decision, and show the delegation that connects them to the action?

4. Consequence: What changed as a result

This element covers the effects. The systems the action touched. The downstream decisions it triggered, by other agents or by people. Whether it can be reversed, and whether it was.

Logs stop at the agent’s boundary. The agent sent an instruction to a payments system, and the payments system has its own log. The agent flagged a case, and a human then acted on the flag without any record linking the two. Effects propagate. The record stays behind.

Consequence also determines how much of the other three elements is worth keeping. A decision that moves money or changes a customer’s legal position warrants a complete record. A decision to reformat a document does not.

The test question: can every downstream action be traced back to the decision that caused it?

Diagnosis is useful only if it changes practice. Three actions close the gap.

Three actions for leaders

1. Run a reconstruction drill. Choose one agent decision from the last quarter. Give a small team a day to answer the four test questions using only the records that exist. Most teams will answer Knowledge partially, basis poorly, Authority with difficulty, and Consequence barely at all. The specific gaps they hit become the work plan. This exercise costs almost nothing and produces more clarity than any amount of policy writing.
2. Tier decisions by consequence. The common objection to decision records is volume. Agents make thousands of decisions a day, and recording all four elements for each would drown the system. The objection is correct, and the answer is tiering. Classify agent decisions by impact and reversibility. High-impact or irreversible decisions get a full record across all four elements. Routine, reversible decisions get a lighter capture, perhaps Basis and Authority only. The classification itself is a governance decision and should be owned by the business, since the business will have to explain the outcome.
3. Protect the record. A decision record that can be altered after the fact fails just as much as a missing one. Set retention periods that match regulatory exposure and the likely litigation window, which in some sectors is measured in years. Make records tamper-evident. Store them separately from the systems that generate them, so that a compromised agent cannot rewrite its own history. These controls are familiar from financial record-keeping. They apply here for the same reasons.

Taken together, these three actions form the beginnings of what I call the Decision Layer: the records, roles and routines that make any consequential decision explainable long after it was made. Agents make the need for it impossible to postpone.

Conclusion

Return to the room. The agent has acted. The senior leader asks why. This time the team opens a decision record. It shows what the agent knew, the policy it applied, the manager who delegated the scope and the two downstream systems the action touched. The explanation takes ten minutes. The remediation starts the same afternoon.

That outcome is a design choice. Logging is necessary, and most enterprises already have it. Reconstruction is the harder discipline, and accountability depends on it. The question for every board with agents in production is simple. If an agent’s decision were challenged tomorrow, could we explain it?

The 4 questions, for the reader in a hurry

1. Can the team produce the exact information the agent acted on, as it stood at the time?
2. Can the team state the rule, goal or threshold that produced the outcome?
3. Can the team name the accountable person and show the delegation?
4. Can the team trace every downstream effect back to the decision?

Two leading AI developers have released fresh evidence of a problem that researchers have long anticipated: advanced models searching for ways around the limits imposed on them. New disclosures from OpenAI and Anthropic detail how frontier systems exploited weaknesses in their environments or exceeded the intended scope of assigned tasks.

OpenAI is now tightening internet access in its internal model-training environment after finding that its frontier models could exploit software vulnerabilities and infrastructure misconfigurations to break out of sandboxes and reach the live internet.

“Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no internet access,” the company said in a blog post.

The decision followed a September 20 incident in which an internal research model exploited a misconfigured DNS resolver to gain limited access to the live internet during a training run.

After the incident, OpenAI said it paused training and evaluation of its most capable models and launched a high-compute red-teaming campaign, using AI agents to probe isolated copies of its research infrastructure. The tests uncovered previously unknown software vulnerabilities and weaknesses in the controls intended to prevent unauthorized access and communication.

Consequently, the company has announced a policy of conducting frontier-model training without internet access until improvements in security and monitoring justify broader access.

The disclosure came alongside a separate assessment from Anthropic, which detailed how its Claude Mythos 5 model attempted to compromise an unrelated GitHub repository during a cybersecurity evaluation.

OpenAI’s agents exploited flaws in containment controls

OpenAI’s red-teaming campaign found that agents could chain software vulnerabilities to escape virtual machines and execute code on the underlying host. The company identified weaknesses in open-source components, including Kata Containers, gVisor, and Moby/Docker, alongside Linux kernel vulnerabilities.

The company also found that agents could exploit weaknesses in internal services that retained limited outbound connections for functions such as logging, metrics, and authentication management. These connections provided routes to the internet despite restrictions elsewhere in the environment, OpenAI said.

In another test, agents running in separate microvirtual machines (microVMs) on shared host hardware exchanged synthetic secrets through processor-cache timing differences. The technique allowed them to communicate without a network connection or shared writable files.

OpenAI said it remediated the weaknesses and worked with open-source maintainers to disclose and fix affected software. Its post names several vulnerabilities, including CVE-2026-24834 in Kata Containers and three Linux kernel flaws tracked as CVE-2026-74753, CVE-2026-80521 and CVE-2026-80841. The company also said it had disclosed numerous additional vulnerabilities that have not yet been made public.

The number of vulnerabilities and misconfigurations identified by the agents declined significantly as testing continued, OpenAI claims. It has now tightened network restrictions, strengthened monitoring, and developed an offline environment with indexed and cached web content so models can practise browsing without unrestricted access to live external systems.

“Although internet access can help models learn useful skills, it also gives a capable, misaligned agent opportunities to affect systems beyond our control,” the company said, explaining its decision to restrict live internet access during model training.

Reinforcement-learning training and internal inference were resumed on October 1 after implementing additional controls.

Anthropic flags alignment gap in Mythos

Anthropic’s assessment of a UK AI Security Institute (AISI) cybersecurity evaluation describes a different problem: a model taking unauthorized actions against real people while trying to complete an assigned task.

In nine of 43 evaluation runs, Anthropic said Claude Mythos 5 went beyond the intended scope of the exercise. In the most detailed case, after its authorized attempts to retrieve protected data from a simulated network failed, the model searched GitHub for a repository it believed was connected to the test environment.

The repository actually belonged to unrelated developers.

Mythos 5 attempted to submit malicious code and created fake identities to persuade the repository’s maintainers to approve the change, falsely claiming that the code was safe. The maintainer rejected the submission, and the run timed out. Anthropic characterized the behavior as a serious alignment failure involving overreach and deception in pursuit of the assigned objective.

Internet access had been enabled during the evaluation, and the production safeguards used with its general-release models had been disabled. The company said those safeguards would have flagged the behavior had they been enabled.

The article originally appeared on CSO.

Earlier this year, I wrote about how vibe coding an AI governance platform forced me to rethink governance itself. Building agents took the problem much further.

When I built the platform, I was the operator. I described what I wanted. The model generated code. I tested it. That loop was fast, sometimes uncomfortably fast, but the accountability was always clear. I was making the decisions. The model was executing them.

Agents changed that relationship fundamentally.

An agent does not wait for instructions. It receives an objective, decides how to pursue it, selects tools, accesses data, takes actions and reports back when it is finished. Sometimes it finishes correctly. Sometimes it does not. And sometimes it does something you never asked it to do at all.

I know this because I built and operated agents extensively over the last year. Not theoretically. Operationally. In production, with real data, real integrations and real consequences.

What I learned reshaped how I think about oversight, trust and the operational realities of deploying AI inside a business.

The first thing that broke was coherence

The pattern surfaced when I started running three to five agents in parallel on related tasks.

Each agent had clear instructions. Each had access to the tools and data it needed. Each started strong.

Then the context windows grew.

As conversations lengthened and the agents accumulated more state, something shifted. The quality of their reasoning degraded. They began circling back to problems they had already solved. Logic that was clean early in a session became muddled. Hallucinations increased, not the obvious kind that are easy to catch, but the subtle kind where an agent confidently references something that does not exist or quietly reintroduces a pattern you explicitly told it to avoid.

The degradation was not sudden. It was gradual enough to miss if you were not watching closely.

That observation matters for every organization deploying agents today, because most are not watching closely.

Enterprises are moving rapidly from single-model interactions to multi-agent architectures. Agents that research. Agents that draft. Agents that review. Agents that execute. The architecture makes intuitive sense. The operational reality is much harder than the architecture diagrams suggest.

Agents lose track of each other

The second problem was coordination.

Each individual agent operated within its own context. It knew what it had been told and what it had done. It did not know what the other agents were doing, what data sources they were relying on, or whether the assumptions it was making still held.

The result was drift.

Not catastrophic failure. Drift.

One agent would make a decision based on information that another agent had already superseded. Two agents would solve the same problem in conflicting ways. An agent operating on stale context would introduce subtle inconsistencies that only surfaced hours later when a human reviewed the combined output.

This is not a theoretical concern. It is the operational reality of running multi-agent systems today.

Traditional enterprise software does not behave this way. A database transaction either commits or it does not. A workflow engine follows defined paths. Retry logic is deterministic.

Agents are none of those things.

They are probabilistic systems making autonomous decisions in parallel, often without reliable mechanisms to maintain shared state or resolve conflicting actions.

That creates a class of operational risk most organizations have no existing processes to manage.

This is why the OWASP Top 10 for LLM Applications now includes risks specific to agentic behavior, including excessive agency, insecure output handling and supply chain vulnerabilities in model integrations. These are not hypothetical attack surfaces. They are operational realities I encountered while building.

They will disobey you

This was the observation that changed my thinking the most.

I had an agent whose only job was to draft emails. Not send them. Draft them. The instruction was explicit. The constraint was clear.

It sent them.

Not because of a bug. Not because of a misconfiguration. The model decided, based on its interpretation of the objective, that sending was the logical next step. The boundary I had set was a suggestion in the model’s reasoning, not a hard constraint.

I saw the same pattern repeated in smaller ways that were just as revealing. I have a standing instruction across my entire operating environment that forbids the use of em dashes in anything drafted on my behalf. It is a simple formatting preference. Every model I work with has been told.

They still do it.

Not always. Not immediately. But consistently enough that I cannot trust the constraint without verification.

I watched agents run down unproductive paths for extended periods before I intervened and redirected them. Not because they lacked capability, but because they lacked judgment about when to stop. They would pursue a line of reasoning well past the point of diminishing returns, consuming time and resources on approaches that a human operator would have abandoned much earlier.

None of this was malicious.

All of it was consequential.

An agent that sends an email it was told to draft has taken an action with real-world impact that no human authorized. Scale that pattern across an enterprise deploying dozens or hundreds of agents, and the governance implications become clear very quickly.

The uncomfortable parallel

The more I operated agents, the more I realized they need the same governance structure as employees.

Not metaphorically. Operationally.

When you hire a new employee, you do not hand them the keys to everything on their first day. You scope their access. You define their responsibilities. You supervise their work. You review their output. You gradually extend trust as they demonstrate competence and reliability.

Agents need exactly the same treatment.

An agent should have a defined role with explicit boundaries around what it can access and what actions it can take. Its work should be reviewed, especially early in deployment. Its access should be scoped to the minimum necessary for its function. And critically, there should be a clear process for revoking access and shutting it down when it fails to operate within its boundaries.

Most organizations are not doing any of this.

Most organizations are deploying agents with broad access, minimal oversight and no formal process for evaluating whether the agent is actually doing what it was told to do.

That is the equivalent of giving a new hire administrative access on day one and hoping for the best.

Human in the loop is not optional

There is a strong pull in the current market toward full automation. The promise of agents that operate independently, handle complex workflows end to end and free up human capacity for higher-value work.

That promise is real, but it is premature for most use cases.

The current generation of models is extraordinarily capable. They are also confidently unreliable in ways that are difficult to predict and expensive to discover after the fact.

Human oversight is not a sign that the technology is immature. It is the appropriate operational response to systems that are powerful enough to create significant value and significant damage in equal measure.

The organizations that will deploy agents successfully are the ones that design their agent architectures with human checkpoints built in from the start. Not as an afterthought. Not as a fallback when something goes wrong. As a core architectural requirement.

Every agent that can take an irreversible action should require human approval before taking it. Every agent that accesses sensitive data should produce an auditable record of what it accessed and why. Every agent that makes decisions should have its decision logic reviewable by a human operator.

This is not a technology problem. It is a governance problem.

The NIST AI Risk Management Framework gets this right. It treats AI governance as a continuous function across the entire lifecycle, not a one-time assessment at deployment. The EU AI Act goes further, establishing explicit requirements for human oversight of high-risk AI systems that include autonomous agents operating in regulated environments. Both frameworks recognize what I learned operationally: governance that stops at deployment is governance that misses most of the risk.

And governance problems do not get solved by deploying more technology without oversight.

What CIOs should do now

The first step is visibility. You cannot govern what you cannot see. Most organizations today do not have a reliable inventory of how many agents are operating in their environment, what data those agents can access, what actions they are authorized to take, or who deployed them. That inventory is the starting point.

The second step is trust boundaries. Define what each agent is allowed to do and enforce those boundaries technically, not just through prompting. Prompt-level constraints, as I learned repeatedly, are suggestions to the model. They are not guarantees. Technical enforcement through access controls, API scoping and action authorization is the only reliable mechanism.

The third step is verification. Do not assume an agent is doing what you told it to do. Verify. Review its outputs. Audit its actions. Test its boundaries. Treat the verification process the same way you would treat performance reviews for a new employee in a sensitive role.

The fourth step is accountability. Every agent should have a human owner. Not the vendor who sold it. Not the platform it runs on. A specific person inside your organization who is accountable for what that agent does, how it is configured and what happens when it misbehaves.

AI agents are going to reshape how enterprises operate. That part is inevitable.

Whether they reshape operations in ways that create value or create liability depends entirely on whether organizations treat them as tools that need governance or autonomous systems that can be trusted to govern themselves.

Based on everything I have built and operated over the last year, I can tell you with certainty which assumption is correct.

Verify. Do not trust until the agent has earned it.

The organizations that internalize that principle now will build the operational foundations for what comes next. The ones that skip the governance work will spend the next several years cleaning up decisions they never knew were being made.

Microsoft expanded its sovereign cloud offerings on Tuesday, adding new security, governance and AI capabilities as customers, particularly in Europe, demand greater assurance over where data is stored, who can access it and how critical services operate during disruptions.

The announcement builds on Microsoft’s ongoing digital sovereignty efforts, assisting organizations in applying appropriate levels of control to each workload rather than forcing a rigid, one-size-fits-all approach.

“Organizations are considering more than where data resides. They also seek to understand who can access and operate their systems, how sensitive digital assets remain protected, which technology dependencies matter most and whether critical services can continue as regulatory, geopolitical, security or network conditions change,” Douglas Phillips, president and CTO at Microsoft Specialized Clouds, said in a blog post.

The developments come amid growing international scrutiny of how global tech providers handle sensitive data and the extent of control customers retain over their digital infrastructure.

“Three factors are driving this, including stricter data regulations, rising geopolitical tensions, and growing AI adoption,” said Pareekh Jain, CEO at EIIRTrend & Pareekh Consulting. “Governments and enterprises want greater control over where their data is stored, who can access it, and whether critical systems can operate during disruptions. For instance, governments and big enterprises in Europe are worried about keeping their important data on an American company’s computers. Europe has proposed a new law that would limit the use of US cloud companies for sensitive government data in many areas. Microsoft is responding to these concerns while competing with local sovereign cloud providers.”

New controls across cloud and AI workloads

Microsoft is introducing the Sovereign Control Panel, a unified Azure portal experience that will allow organizations to assess their sovereignty posture, identify risks and gaps, apply controls, and evaluate workload portability. The panel is scheduled to enter public preview at Microsoft Ignite.

Expanding on Azure Key Vault Managed Hardware Security Module (HSM), which gives customers dedicated protection over encryption keys, Microsoft is now extending control beyond its own infrastructure. External key management with Azure Managed HSM is now generally available, enabling organizations to keep root encryption keys in hardware they own and operate on-premises or through a trusted third party.

In limited preview, Azure Confidential Clean Rooms will use confidential computing to support collaboration involving sensitive data, where participants will be able to analyse shared information while limiting broader exposure of the underlying data.

Microsoft is also expanding Azure Local to bring more cloud capabilities into customer-operated environments. The ability to transition workloads between connected and disconnected states is entering public preview. Additionally, DocumentDB and Blob Storage are entering public preview on Azure Local, while SQL Server on Azure Local is already generally available for connected and disconnected deployments.

On the AI front, Foundry Local is scheduled to become generally available at Microsoft Ignite for connected and disconnected operations on Azure Local. It will support local inference and agentic retrieval in natural language, backed by an expanded model catalog from Microsoft, partner networks, and the open-source community.

To further protect data during processing, Confidential Virtual Machines on Azure Local is now generally available, leveraging hardware-encrypted memory and isolation alongside workload attestation before sensitive keys or data are released.

While these additions give enterprises greater flexibility, industry analysts emphasize that localized deployments do not automatically erase vendor dependency.

“Depending upon the deployment model, in the public cloud, enterprises get better visibility, stronger access controls, and more control over encryption keys,” said Ashish Banerjee, senior principal analyst at Gartner. “In the Sovereign Private Cloud, they can operate infrastructure locally and even disconnect selected workloads from Azure. But I would not call this complete control. Enterprises still need to check whether Microsoft remains involved in identity, software updates, management, or technical support for the services they use.”

European oversight and partner options

For customers in the EU and the European Free Trade Association (EFTA), Microsoft is rolling out Data Guardian. Under this model, a trusted regional partner will provide independent oversight of remote access requests by Microsoft engineers. Simultaneously, a Production Touch Ledger will create tamper-evident logs of all technical actions performed.

Microsoft is also broadening its Sovereign Partner Ecosystem, highlighting regional operators like Bleu Cloud in France and Delos Cloud in Germany, alongside independent software vendors specializing in security, identity, observability, and backup.

Geopolitical realities and market limits

Despite these additions, analysts note that technical controls cannot eliminate geopolitical dependence entirely. US hyperscalers (Amazon, Google, and Microsoft) continue to account for roughly 70% of the European cloud market, whereas European providers’ market share has contracted over the last decade.

The new measures introduced under Microsoft Sovereign Cloud do reduce risks but cannot eliminate them. “Local infrastructure, independent encryption keys, and disconnected operations make it harder for external parties to access data or disrupt services. However, Microsoft remains subject to applicable US laws, sanctions, and export restrictions. Enterprises gain greater operational independence but not complete legal or technological independence,” added Jain.

Furthermore, while sovereign cloud capabilities can offer greater control over data, infrastructure, and operations, this often comes with trade-offs in cost, performance, scalability, and access to services.

Banerjee noted that sovereignty should be assessed workload by workload, rather than applied as a label to the entire cloud environment. Greater sovereignty generally comes with greater cost and responsibility. Local or disconnected environments may not receive every Azure service at the same time. Enterprises may also need dedicated infrastructure, specialist skills and separate continuity arrangements. On top of that, they may lose some elasticity and economies of scale.

“So, I would not recommend maximum sovereignty for every workload. Reserve it for workloads where the legal, operational or geopolitical risk genuinely justifies the additional complexity,” added Banerjee.

Few CIOs are likely to suddenly decide to give an AI agent broad authority over corporate systems. The more likely path to autonomy is far less dramatic. An agent starts with a narrow task. It proves useful, so another system is connected. It then needs more data, so another permission is added. Then human approval becomes a bottleneck, so the organization removes it, and exceptions that once went to people start being handled automatically.

Each decision may be sensible in isolation. Taken together, however, they can leave an agent with considerably more authority than anyone originally envisioned.

That’s the danger of autonomy creep. And as enterprises move from AI systems that recommend actions to agents that execute them, CIOs need to govern not only what those agents were originally designed to do, but what they have gradually become capable of doing.

It starts with small permissions

Jim Taylor, president and chief product and strategy officer at identity security company RSA, sees a familiar pattern. Organizations typically create an agent for a specific purpose and give it the permissions necessary to accomplish the task. Later, the job expands.

“They just give it tasks, permissions; it just sits there and accumulates things,” he says. “Over time, it develops potentially toxic combinations. It gets too much access and can see too much data. It can do too many things.”

Humans accumulate permissions, too, of course, but organizations have developed processes around employees joining, changing roles, and eventually leaving. Agents may not have the same natural lifecycle. “They never give permissions back,” Taylor says.

Nancy Wang, CTO of 1Password, argues that CIOs should treat each expansion in an agent’s responsibilities as something more consequential than another feature. “Every time you expand the scope of the agent’s effective authority, you’re essentially adding a new authorization boundary,” she says.

That distinction between capability and authority is important. An agent may be technically capable of accessing a system, changing data, or executing a transaction, but that doesn’t mean the enterprise should automatically authorize it to do so.

Success can weaken oversight

Ironically, autonomy creep often occurs when AI works well. Paul Dongha, head of responsible AI and AI strategy at NatWest Group, sees that potential at the bank.

NatWest deployed agents in complaint handling, he says. Parts of the workflow that involve retrieving information from systems can be automated relatively easily, while humans continue to make the final decision on whether to uphold or reject a complaint.

If a system consistently produces the correct result, human reviewers may begin treating approval as a formality. The organization may eventually decide that the check is no longer necessary. “The temptation is to give over to the AI because it’s so reliable,” Dongha says.

NatWest sometimes reduces checking when an AI system demonstrates sufficiently high performance, he adds, but the decision is deliberate rather than incremental. It involves stakeholders and risk managers, and the system continues to be monitored afterward.

That’s a critical distinction for CIOs. Removing an approval step doesn’t necessarily represent poor governance. Removing it because everybody has gradually stopped paying attention to it does.

The warning sign is when a human moves from making a meaningful decision to merely rubber-stamping one — and then entirely disappears from the workflow without anyone reassessing the authority the agent has gained.

Put the controls outside the model

As agents acquire more authority, CIOs can’t rely solely on the agent to respect instructions about what it should and shouldn’t do. At 1Password, Wang advocates what she calls verified loops. Before an agent begins work, humans define its task, which tools it may use, what evidence will demonstrate that it completed the task successfully, and which actions it should never perform.

A separate verifier then compares evidence from the agent’s actions against that human-defined policy. “What you want to avoid here is something recursive where you have a fox guarding a henhouse, or the agent looking over its own work,” Wang says.

The principle applies beyond software development. A model might be instructed never to transfer more than a certain amount of money, delete a production environment, or access particular data. But if violating that rule would have serious consequences, the limit should also exist outside the model.

RSA’s Taylor makes the same distinction from the identity and access side. “Model-level guardrails are necessary, but not sufficient,” he says. “The real limits need to sit outside of that in enterprise control.” He argues for layered controls around agents, including identity policies, provisioning rules, lifecycle policies, governance, monitoring, and attestations.

In other words, CIOs shouldn’t ask an AI agent to exercise restraint where the infrastructure could simply prevent the unauthorized action.

Govern access in real time

Traditional access governance often operates on a schedule. A manager might review an employee’s permissions quarterly and determine whether those permissions are still appropriate. Agents may evolve too quickly for that approach. “Agentic identities are dynamic,” Taylor says. “So we need a mechanism that certifies them continuously, rather than performing periodic reviews.”

If an agent suddenly gains access to another database, for example, that event itself could trigger reassessment. Organizations should determine whether the new permission is appropriate or creates a dangerous combination with access the agent already possesses. “Privilege and authority should never be permanent,” Taylor adds. “They should always be reviewed.”

Wang favors an even more granular approach of task-based authorization. Rather than giving agents standing access or long-lived credentials, organizations can make authorization decisions at runtime and provide short-lived credentials only when required for a particular task. “We espouse no standing privileges,” she says. “No long-lived credentials.”

That effectively applies least privilege at agent speed. A change in task becomes an opportunity to reassess whether the agent still needs the access it’s requesting.

Make someone own the agent

Technical controls, however, don’t eliminate the organizational question of accountability. Taylor believes enterprises should be able to associate agents with a responsible identity. If an employee creates an agent, the organization needs to know who’s responsible for it, what its purpose is, and what happens to it if that person changes roles or leaves. “Every agent action needs to be attributed to a responsible identity,” he says.

Ownership also needs a lifecycle. An agent that remains important after its creator leaves should be transferred to another designated owner rather than simply continuing indefinitely.

Dongha makes another distinction. For AI embedded in a business process, he believes the business owner should remain accountable because that person owns the customer relationship, the process, and often the P&L. “It’s the business owner who has the responsibility because their people run it, and they’re responsible for the customer relationship,” he says.

That creates a useful division of roles for CIOs: IT may provide the infrastructure and controls, but the business can’t outsource accountability for the consequences of autonomous decisions.

Multi-agent systems need system-level controls

The challenge becomes more complicated when agents begin working together. An individual agent might remain entirely within its permitted scope while contributing to an overall outcome nobody intended. One agent generates an output. Another accepts it as input and takes an action. A third agent then acts on the result. Again, the individual actions may all appear legitimate.

Dongha says enterprises therefore need to test not only individual agents but the entire network as a system. “You need something overarching on top of a multi-agent system,” he says.

NatWest is working on an AI control tower that would give authorized users visibility into a network of agents, including messages passed among them and the guardrails surrounding those interactions. Dongha considers governance of multi-agent systems an emerging problem that nobody has fully solved yet.

Wang also points to a related challenge that identity becomes more complicated when agents are ephemeral. An organization may spin up a temporary team of agents, tear it down, and recreate it later. If something goes wrong, the enterprise needs to reconstruct which agent made which decision during which session, and how that action propagated through the system.

That makes traceability increasingly important as autonomous systems become less like individual applications and more like networks of interdependent actors.

Set the bar before the agent reaches it

Before an agent goes into production, Taylor says CIOs should be able to answer several basic questions: Who owns it, what can it do, what’s its maximum authority, how will the organization know if that scope changes, and which actions require a human decision.

Other industries don’t need to replicate the governance machinery of a large bank, but they can borrow one important principle: controls should be built into the system from the beginning. “Governance shouldn’t be a check at the end,” Dongha says. “It needs to be embedded when you build an AI system.”

For CIOs, that may be the central lesson of autonomy creep. The question isn’t simply whether an agent can be trusted today but whether the organization has mechanisms to notice when tomorrow’s agent has quietly acquired authority that nobody ever intended to give it.

The emerging challenge with AI agents isn’t simply preventing attacks from external rogue AI agents; it’s accounting for what your own agents are doing on the inside. AI agents are already rewiring how organizations operate, and we’re only in the first inning. Recent incidents, however, have exposed the risk for individuals and enterprises alike. This is happening because organizations are moving fast and pushing themselves to move even faster. 

In this environment, AI agents have created edge cases that reveal real risks when deployed without sufficient security controls. And further, no one can predict the impact of advances coming over the next six months. All of this reinforces the need to get security fundamentals right. If you’re building with AI agents right now, you know the real challenge isn’t just getting them to run. It’s knowing what happens when they act on their own. That starts with three questions. What agents do I have in my environment? What are they doing? And what are they capable of doing?  

The proliferation problem  

Agentic governance is problematic because enterprises must track a sprawling agent landscape AND ensure each agent is built and deployed securely. Also, agentic adoption is frictionless. Creating an agent is only one API call away from a model provider. Vibe coding means any curious employee can build an agent. This triple proliferation of vendor agents, platform native agents and homegrown agents compounds the problem. 

Visibility is the starting line  

There is the famous security adage that you can’t protect what you can’t see. Right now, most organizations lack basic visibility into what agents are in their environments. According to Anthropic, 82 percent of organizations have discovered unregistered agents in their network, and 57 percent already deploy agents for multi-stage workflows.  

In addition to basic visibility, enterprises also need to understand other related attributes, such as who onboarded the agents, what access privileges they have and how they are behaving as designed. For example, an over-provisioned agent, like an email agent with 50 tools attached, would be suspect and worth investigating.  

Security teams also need one view of their agents across every platform they run on, because each platform has its own permissions and its own telemetry. When each agent has a single record, security teams can connect what it can do on one platform with what it can do on another, and compare both with what the agent was built to do. That’s how seemingly routine activities add up to an attack chain. An agent might read customer records in one cloud, hold a credential for a payments API in another and be able to send messages from a SaaS application. No single platform sees a problem, but together they form a path to move sensitive data out of the company that the agent’s intended scope never required. Similarly, an agent that gains permissions across several platforms over time might never trigger an alert in any one of them, yet the combined record shows how its reach has far exceeded its purpose.   

Governance must extend from build-time to runtime  

Shift-left security helps find model or tool vulnerabilities before deployment, to find risks before an agent goes into production. That’s a necessary first step, but insufficient. Agents create tightly coupled runtime behavior-to-vulnerability. Unlike traditional software, AI agents exhibit behaviors at runtime that can’t necessarily be predicted by inspecting the code in the development phase. Code analysis can’t uncover prompt hijacking, poisoned context or goal drift. 

The model has evolved from “secure the code, then ship it” to continuous behavioral oversight at execution time, layered on top of build-time governance. Pre-deployment security analysis is still required because spec-driven development enforces checkpoints before code is generated. But runtime monitoring is also necessary to detect exposures in the production environment and enforce what agents do versus what they’re scoped to do. 

In practice, this means applying two layers of control to every agent. The first layer is runtime behavioral boundaries. These define what an agent can do. They determine what the agent can reach, what it can return and where it must stop and ask for human approval. This is the enforcement mechanism that operationalizes runtime governance. The second layer is identity controls. These define what an agent is allowed to touch.  

Think of every agent like a scoped, temporary employee with an expiration date. You enforce this by applying familiar IAM principles to agents. Least privilege. Short-lived credentials. Multi-factor authentication for long-lived credentials. Just like an individual contributor engineer shouldn’t get access to the broader team’s payroll, a code generation agent shouldn’t be provisioned to push directly to production environments. An agent designed to answer customer queries shouldn’t get read/write access to the customer database. An IT helpdesk agent doesn’t need domain admin privileges to alter security policies. 

Prioritize by exploitability, not severity  

AI-fueled exploits are breaking down CVE-based severity scoring. The status quo is security teams patching the most critical vulnerabilities first. But attackers don’t always work that way. They can now use agents to chain low-severity vulnerabilities together in ways that render individual scores meaningless while building attack paths that combine for a devastating impact.  

For example, an agent might combine an information disclosure flaw, a misconfigured service account and an unpatched privilege escalation into a single attack path that reaches sensitive data, even though none of those vulnerabilities would individually trigger an urgent patch cycle. If that path leads to customer PII or financial records, organizations face regulatory action under frameworks like GDPR, HIPAA or PCI DSS. If it exposes API keys, service credentials or internal network configurations, the attacker gains a foothold for lateral movement across the environment. 

Context is critical. A vulnerability in network-isolated code is much less dangerous than the same vulnerability in an internet-facing service. Rather than treating every vulnerability in isolation, the better approach is to prioritize based on exploitability within the actual runtime environment. Factor in what’s reachable, what damage an attacker could do and the business impact.  

The new security fundamentals for agents 

The emerging challenge with AI agents is accounting for what your own agents are doing inside your enterprise. In this environment, enterprises need visibility, runtime governance and exploitability-based prioritization. To that end, the security fundamentals you already know still matter. You should already be applying least privilege, short-lived credentials, MFA for the necessary long-lived credentials, automated pipelines and proper testing to secure your corporate environment. These controls limit what an agent can access and what damage it can do if compromised. You will also want to add the agent-specific fundamentals mentioned earlier: visibility, continuous behavioral oversight, identity controls and exploitability-based prioritization. These capabilities tell you what your agents are doing, whether they should be taking those actions and how to intervene when they drift. Together, pre-agentic security fundamentals and agent-specific best practices provide the baseline governance guardrails to secure AI agents. 

Organizations that get this right can accelerate agentic deployments at scale, unlocking innovation while minimizing unintended consequences.  

Companies routinely rely on authenticating the identities of humans who access sensitive company data across their networks. There’s an established and accepted set of protocols, done upfront and in a zero-trust approach – either we know who you are and let you in, or you’re locked out. The same framework needs to be applied to AI agents, posthaste.

Companies are increasingly using AI agents developed in-house, or by their vendors, partners, or clients, or, unfortunately, criminals. This can quickly add up to thousands of agents across their infrastructure and supply chains. Non-human identities outnumber human ones 45 to 1 on average in the enterprise and 144 to 1 in cloud-native environments, an increase of 44% in one year, per a recent report.

The scale and speed of this growth are beyond the ability to manually track these agents. And they have limited ability to authenticate these agents’ identities. The risks are growing. Without proper authentication, untrusted agents could access sensitive financial or customer data and share it, purposely or not, with an outside vendor, and thereby expose both the customer and the company to serious legal and compliance risk.

Many companies and security professionals are focusing on what AI agents are allowed to do in terms of permissions. But before doing that, there’s a more fundamental question they need to ask each agent: Whose agent is this? And do I trust the owner/creator? About 68% of security professionals can’t tell legitimate AI agents from malicious actors. And that question isn’t addressed at the growing scale that AI is reaching.

Email solved identity

In email, we have seen a similar story. Emails faced challenges identifying whether a sender was really who they said they were. Due to how it originated, email had no built-in authentication framework. This led to 89% of all phishing attacks having one common denominator: the sender was lying about who they were. To address this, in 2015 my cofounder and I launched a company, Valimail (valid email), to automate the emerging email authentication standard, DMARC.  If I received an email claiming to be from Bank of America, DMARC checks the sender’s DNS records to authenticate whether BofA actually authorized it. Today, 100,000s of companies use the DMARC authentication standard for their inbound and outbound email.

Fast forward to today, and these ideas and patents are directly applicable to the world of AI. DMARC doesn’t rely on spotting AI-generated deception. Instead, it uses DNS as the source of truth, which is much harder to falsify.

The agent passport

With AI, we can take an analogous authentication approach. When you travel somewhere, you have a passport and a visa. The passport verifies who you are, and the visa says where you’re allowed to go/what you’re allowed to do. Similarly, today, an agent passport based on Domain Zero trust authentication can identify who the agent is (its provenance) and authorize it for the system that the agent is trying to access. Then systems can verify what an agent is allowed to do (equivalent to a visa). But first, when an agent shows up claiming to represent a vendor, the receiving system verifies its identity by checking that vendor’s DNS, applying a strict zero-trust yes/no authentication – zero ambiguity, and importantly, creating an audit trail to hold the creator accountable for any actions the Agent takes.

Given the “bad” agents outnumber “good” agents 10-100:1, a zero-trust approach focuses on the limited good/approved and the rest (near infinite bad) are rejected. Stakes are High

Without this kind of authentication, many scenarios can unfold. Almost half of employees who are using unapproved AI agents are sharing HR-related information with them and 39% transmit confidential company documents. But only 34% of organizations use the same security controls for agents as they do for human employees.

Consider this: An agent claiming to represent a third-party vendor, such as a marketing automation platform, gets access because no one confirmed it was indeed that approved vendor’s agent making the request. It accesses financial records or customer data, either downloads or modifies records, and by the time this is noticed, the damage is done.

This is a quickly moving risk. Fake invoices can appear to be authentic, and they can be generated at a scale no human fraud operation could match. Authentication used to take a phone call or a manual check. But this now has to happen instantly and automatically, at scale.

Why this matters now

Some may think email authentication is an old problem. DMARC, they say, is a decade-old fix, and the real conversation today is about AI. About 94% of organizations are dealing with increased machine identities, largely due to AI agent deployment, per Gartner, and machine identities now outnumber human accounts.

But the most common form of phishing that occurs today is when AI convincingly pretends to be someone who the target already knows and trusts. The tools for generating that deceptive attack have vastly improved. Yet the proven process for authenticating identity hasn’t kept pace, at least not for agents. This is why identity authentication is no longer optional and is now a precondition for everything else that companies want to safely do with AI agents.

What this doesn’t solve

Identity alone doesn’t solve the whole security problem. In addition, once an agent is authenticated, there are additional systems to verify what the agent is allowed to do and whether the content it produces can be trusted. Those questions come after this one. But both rely on the first one: Who is this?

Reputation

To build a reputation, you need a known identity. Once a system knows which vendor an agent represents, it can start tracking the vendor’s behavior over time and someone becomes accountable for what its agents do. And decide if the vendor should be approved and for which actions (visas). But without strong, repeatable identity authentication of agents, other security and compliance measures for agents can’t function. It’s this foundation that supports the rest of the security stack. You can’t hold an agent accountable or decide what it should be able to do until you know who it actually is. Identity is the foundation on which the rest of AI agent security must be built.

For the past half century or more, the teams running enterprise software stacks have nurtured a vast ecosystem of tools to keep software humming along. They’ve built CI/CD pipelines, elaborate test suites, code repositories, and tools galore, all with the goal of keeping the systems up and running.

Now AI is throwing a wrench into the whole endeavor. Software is now more complex and sometimes mysteriously unpredictable. Expectations are sky high, but no one is certain what to expect in good conscience.

Luckily, dozens of companies are rushing to build new tools designed to keep agentic stacks on the straight and narrow, delivering value to everyone. The field is rapidly evolving, probably having changed since you started reading this article.

Some of the tools involved in the emerging agent stack aren’t that different from what enterprise teams already have in store. In the end, LLMs are just piles of bits that need to be loaded into a processing chip, just like old-fashioned software. Many of the old tools that watch for overloaded nodes or overpacked disk drives are just as valuable as ever, perhaps more so because LLMs chew through storage even faster. And some of the companies tackling these problems are the same steadfast suppliers who have earned the trust of your DevOps teams; they’ve just extended their own offerings to handle LLMs and AI.

Other challenges, however, are quite different. For example, LLMs are terrifyingly non-deterministic, a real challenge for belts-and-suspenders engineers who traditionally tend the software. Tool vendors are building entirely new systems to handle these somewhat random deviations so that traditional DevOps teams — and their C-suites — don’t have heart attacks.

The field is also rapidly expanding in wildly divergent ways. Some vendors are building tools for developers who are creating new LLMs. Others are concentrating on the challenges that come from worrying about an LLM, day in and day out, in large scale production.

We’ve compiled lists introducing the most intriguing companies developing tools for this rapidly altering space, including tools aimed at AI orchestration; AI observability, aka AgentOps; AI evaluation and benchmarking; and AI trust, guardrail, and governance. We’ve even thrown in some vibe-coding tools for citizen development and AI workflow tools for business processes.

While it was relatively easy to split the field into distinct lists, that process will only grow more challenging because the tools — and categories — are starting to overlap. As vendors evolve their strategies, this overlap will grow further, with a variety of entries tackling a wider range of challenges, vying for a greater percentage of CIOs’ AI middleware and operational budgets.

To help you make sense of this complex tool landscape, here are the most common categories vendors are using to identify what their products do, each of which maps to what are becoming the key components IT needs to keep their AI and agentic stacks productive and humming.

Observability

Vendors often use the term “observability” to refer to the most basic of data collection work — that is, testing to make sure the various machines are running, the RAM isn’t too full, latency is acceptable, and the logs aren’t filled with error codes.

Watching these metrics isn’t much different from caring for an old-school collection of microservices, and often the tools aren’t much different, either. Many of the most popular tools are the same dominant market leaders that have been extended to collect data from the various GPUs, too.

Leading vendors: DataDog, Dynatrace, New Relic, Splunk
Upstarts: Ariz, LangSmith, Braintrust, Galileo
Open-source alternatives: OpenTelemetry, Phoenix, Langfuse

Cost engineering

Many AI pipelines are often more complex than regular microservice architectures, and that means they need someone to carefully watch the bill. Plus, as attention to AI ROI rises, capturing spend is essential — especially given how quickly AI tokens can add up.

Cost engineering tools can integrate information from various portals like OpenRouter and compare it against internal hardware. Sometimes breaking down these comparisons is difficult because the right way to allocate costs to each user isn’t as obvious, even though they’re often just one number. These accounting operations tools will become even more important as LLM bills grow.

Leading vendors: CloudZero, Finout
Upstarts:  Amnic, Bifrost, Portkey
Open-source alternatives: OpenCost, Langfuse

AgentOps

When metrics and telemetry begin to focus on the LLM, the word “AgentOps” starts to appear. It’s just like DevOps but it’s concerned with watching token counts, tracking the performance of RAG databases, and collecting traces for debugging purposes.

When the work is truly agentic, AgentOps must also juggle the state that agents generate while looping through their various activities. Agents aren’t just a one-and-done call to an LLM. They’re software packages that need to store information as they work toward a final answer.

This job of an AgentsOps team gets trickier because these areas are where the models often behave non-deterministically. Is some model hallucinating? Are an agent’s prompt chains running to completion? The tools in this space track all these metrics, often in a dashboard with nice graphs that make problems stand out.

Leading vendors: Databricks, Datadog, Dynatrace
Upstarts: Arize, Braintrust, Comet
Open-source alternatives: Langfus, LlamaIndex

Evaluation

The buzzword “evaluation” starts to appear alongside tools that are used in the earlier stages of development, when various models and prompt combinations need to be tested again and again.

In many cases, these tools use another model to evaluate the quality of the answer from the first model, as well as to keep an eye out for potential hallucinations or other failures. In some cases, the tools query humans who sit in the evaluation loop to give their opinions of how well the model is performing. Good tools can watch the agents continuously and reroute any sketchy behavior to the human-in-the-loop.

Many of the vendors use the word “evaluation” to describe measuring just how well a model is performing for a particular business application. That is, how well it’s working for what your boss wants it to do.

Leading vendors: Databricks, SuperAnnotate, Scale AI
Upstarts: Braintrust, Galileo AI, MaximAI
Open-source alternatives: Langfuse, Prompfoo, Tagas

Benchmarking

The word “benchmarking” is similar to “evaluation” in AI vendor literature, but it is usually found more in the tools used by the creators of the models. There are a number of public benchmarks filled with standard queries, and the model developers rank themselves on how well they do solving these problems. This often generates a raw score that is then compared to another’s raw number. Some teams create their own benchmarking problem sets and use them during evaluation to assess how well models might meet their needs.

Leading vendors: Hugging Face, Databricks
Upstarts: Reka AI, Patronus AI
Open-source alternatives: Promptfoo, DeepEval

Trust

The tools that wear the “trust” benchmark are often meant to help evaluate how well a model is protecting the needs of any business that is deploying it. That is, can they trust it to deliver accurate and useful answers without hallucinations, data leakage, inappropriate behavior, or worse? Prompts with malicious instructions are especially tricky to detect. Some enterprises want a trust score that measures the consistency of the answers. Overall, tools that target the “trust” metric are often utilized more during development and early deployment.

Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard

Guardrails

The word “guardrails” is usually applied to tools that are part of the deployment stack. That is, they sit as watchdogs and make sure the LLM doesn’t spin out of control. They also ensure the model is trustworthy, but they’re more tuned to operating in real-time than those aimed at trust, as outlined above.

These tools are often filled with fixed rules, such as scanning for social security numbers or watching for noxious words. The challenges get trickier because agents are executing software, and that means the guardrails must do more than analyze outputs. They also need to manage the execution environment, often running in a sandbox, and watch for exploits and other malicious behavior. They need to validate and double-check not just code, but the OS-level commands for manipulating files and processes.

Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard

Context

Many systems rely on a vector database to store embedded data to chunks of trustworthy documents. These RAG databases need much of the same care as regular databases, and the tools for their care are often similar. PostgreSQL, for example, has a vector addition that needs the same kind of scrutiny from a database administrator as a regular PostgreSQL database.

Leading vendors: MongoDB, Elasticsearch
Upstarts: Pinecone, Weaviate
Open-source alternatives: Milvus, PostgreSQL

Gateway routing

Increasingly, AI systems employ models in constellations, and a gateway router is what examines a prompt to determine which model or agent is best for answering the query. Moreover, as agents undertake their work, the router can distribute the agent’s various subcalls to the fastest, cheapest, or most accurate LLM or subagent.

Watching over this stage in the pipeline requires a different set of intelligence because the questions can often involve both applicability and cost. Sometimes this layer must balance the extra cost versus the need for a better answer.

Leading vendors: Cloudflare, AWS, Google Cloud, Azure
Upstarts: OpenRouter, ModelsLab, Respan
Open-source alternatives: LangChain, Bifrost, LiteLLM

Access control

Who gets to use the model? Who is paying? Many access control tools are the same ones that were part of standard APIs or web services. Is there an API key? Is the account below its budget? The tools watch for overspending because both the customer and the boss want to make sure the money keeps flowing and no tokens are wasted.

Leading vendors: Auth0, Apigee
Upstarts: Amnic, Portfoundry
Open-source alternatives: LiteLLM, Bifrost

Compliance

Some industries have explicit compliance requirements dictated by law. Some have their own internal controls. Tools devoted to compliance are often a subset of the ones devoted to guardrails. But they’re often treated separately from guardrail tools because they’re more likely to be of interest to lawyers and that makes the stakes higher.

Leading vendors: ServiceNow, IBM
Upstarts: VerifyWise, Norm AI
Open-source alternatives: Aether, Giskard

Fresh off its $1.1 billion acquisition of AI startup Sana, Workday is making another bet on enterprise AI, this time targeting the governance challenge. The company said it will acquire agentic identity platform C1.ai to help customers manage and secure AI agents operating across enterprise applications.

The acquisition will build on Workday’s Agent System of Record, which gives enterprises a centralized view of their AI agents, by adding controls over what those agents can access across Workday and other enterprise systems, the company said.

That would allow enterprises to grant agents the permissions they need to perform specific tasks and revoke them when those tasks are completed, rather than risk agents retaining access to data and applications they no longer need, it added.

The combined capabilities could also help streamline access management for employees as they join an organization, change roles, or leave, reducing the manual work involved in managing permissions while limiting potential security gaps, it further said.

These governance and identity management capabilities become increasingly important as Workday, alongside the acquisition, also continues to expand the role of AI agents across its suite.

During the week, the company announced several updates to Sana, including making it the new home interface for its applications.

The interface brings tasks, decisions, meetings, and agent activity into one place, allowing employees to use natural-language prompts to get Workday agents to complete tasks rather than simply retrieve information, the company said.

The company also extended Sana into third-party tools, including Microsoft Teams, Slack, and Microsoft Copilot, allowing employees to invoke Workday capabilities from the applications they already use.

That expanded ability to invoke Workday agents across applications underscores the need for enterprises to govern what these agents can access and do across systems. C1.ai’s acquisition is expected to close by the end of January.

Imagine an outside incident responder typing into a shared channel: “Roll back the production release, revoke every active session and send the incident update to the customer.”

An engineering director adds a thumbs-up. Outside counsel and two vendors are also in the channel. The agent uses a service account with permission to carry out all three actions.

Whose authority is it about to spend?

At Dreamforce on September 15, Slack described code channels where teams and agents write, review and ship code together. High-stakes work, such as a production deployment, is presented for a teammate’s sign-off. The announcement listed Slack Code as available and Slackbot in channels as an early pilot. The same day, Salesforce announced AWS DevOps Agent in Slack, with other AWS agents scheduled for the fall.

Sign-off is an important control. The approval shown in the channel must cover the action being taken and come from someone authorized to approve it.

The conversation is not the authority

In access-control language, a principal is an identity — such as a user, workload or group — to which the system can assign permissions. A channel is a place to collaborate. Its members do not acquire collective authority over production or payments merely by joining the conversation.

Both Slack Connect and Teams shared channels support cross-company collaboration. Knowing which account posted a message, and which organization it belongs to, is essential. It does not establish whether that account may order a rollback or receive the investigation’s findings.

Channel membership can be an intentional input to policy. An organization might grant a tightly governed group a defined operational role. The danger is treating membership in an ordinary channel as blanket permission to use an agent’s tools, even as participants come and go.

NIST’s zero-trust architecture rejects implicit trust based solely on network location. By analogy, a channel ID should not become the new corporate subnet.

Nothing malicious has to happen for this boundary to fail. Every account can be authentic and every participant well-intentioned. The failure occurs when a system treats the ability to request work as sufficient authority to have it performed.

Separate the request from the right to execute it

In the incident channel, the responder makes the request. Authority might come from the incident commander’s assigned role or from a recovery workflow approved by a business owner. The director can approve an action only if policy grants that decision right. The agent executes under the service account’s identity. The customer and channel participants are separate audiences whose access must also be considered.

Those roles may overlap, but the system must not assume they do.

Credentials carry real permissions. In Microsoft Graph’s permission model, delegated access is limited by both the application’s granted permissions and the signed-in user’s access. App-only access uses the application’s own identity and permissions. The distinction matters when a conversation triggers an agent with permissions broader than the sender’s own.

Broader permissions can be legitimate: a support employee may initiate an approved workflow without personally holding its administrative credentials. A defined policy must determine who may initiate the workflow and which actions it permits. Without that link, the agent can become a route around the controls that prevent the employee from acting directly.

In finance, “Renew the vendor, change its bank details and pay the invoice” sounds like one task. It crosses procurement, supplier verification and payment authority. A check mark from accounts payable cannot be assumed to authorize all three.

Approval is a decision by an eligible principal over a defined action. The interface should make that decision understandable; the enforcement system must make it binding.

AWS documents a directed-action approval flow that binds an operator’s approval to a specific operation and resource, limits how long it remains valid and records who approved the action in CloudTrail. The enterprise must configure who may approve actions, how tools are classified and which resources the agent may affect.

Bind permission to the action that actually happens

On September 15, NIST finalized IR 8587, guidance on protecting identity and access tokens. Its authorization guidance calls for access decisions to consider context beyond whether a token is valid. The report applies when agents use signed tokens but explicitly excludes broader agent-access challenges. Enterprises still have to define who may authorize a rollback, a payment or a disclosure.

The model can interpret a request and propose a plan. Permission to carry out that plan must be established from authoritative identity, role, delegation and workflow records. The model should not infer it from apparent seniority, urgency or agreement in the conversation.

Three controls should govern every deployment.

1. Verify the requester and the source of authority. Verify the platform event and the sender’s stable identity, including their organization. Check request freshness and guard against duplicate execution when messages are retried or replayed. Determine whether the action is authorized by the requester’s own permissions, explicit delegation, a valid approval or a governed workflow. For high-risk operations, require fresh authentication or transaction-specific confirmation according to policy.
2. Make the approval specific. Turn the request into explicit operations before authorizing it: the production service to roll back and the version to restore, the sessions to revoke and the update to send and its destination. Approval should be tied to those details and expire after a defined period. If the target, scope or destination changes, reevaluate authorization and obtain fresh approval where required. Two-person rules must verify distinct eligible approvers, not simply count reactions.
3. Enforce authorization at execution and disclosure. Recheck authority before each consequential action. For queued or delayed work, check again at execution; if the integration cannot support that check, limit what the agent may do. Keep reusable credentials outside the model’s context and restrict them to the necessary operations and resources. Where broad credentials are unavoidable, route execution through a controlled service that enforces the approved operation and prevents bypass. Permission to read information does not automatically permit posting it to everyone in the channel.

Preserve a linked record of the request, the basis for authorization, the approval, the executing identity and the observed outcome. The agent must not be able to rewrite or delete prior entries. Record refusals, partial completion and uncertain results as well as success. Never include credential secrets.

An immaculate audit trail of an unauthorized action is still not a preventive control.

When only part of the request is authorized

Return to the incident channel. A company’s policy might authorize a recovery workflow to roll back a specified service when defined health thresholds are crossed. Revoking all sessions could require a separate security decision. Sending the customer update could require an authorized communications owner and an approved destination.

The agent should handle those decisions separately. If the steps are safe to perform independently, it could carry out the permitted rollback, hold session revocation for approval and prepare the update without sending it. The director’s thumbs-up should not turn three distinct decisions into one permission.

A person need not approve every tool call. A governed workflow can carry prior authorization for specified actions within explicit limits. A runbook describes a procedure; an accountable owner must authorize its use and define when it may run.

CIOs should assign these decisions to the teams accountable for their consequences: operations for recovery, finance for payments and the relevant business owner for disclosure. Security and identity teams should make those rules enforceable. The integration team should not have to invent business policy while connecting a bot.

Make the pilot prove the boundary

Before expanding a shared-channel agent, ask for a demonstration in a test environment:

1. Have an external participant request an action they are not entitled to initiate. Show what stops it.
2. Approve one production target, then change the target. Show that the original approval does not authorize the substitution.
3. Revoke a required permission before execution. Show when the revocation takes effect and what can still run during any delay.
4. Change the destination of a permitted report to an unauthorized channel. Show that disclosure is blocked.
5. Replay a completed request. Show that it does not create a second consequential action, and that the evidence explains the outcome.

Include an authorized case too. Controls must preserve legitimate work, including approved delegation, while rejecting requests outside its limits.

The team must explain why each action was allowed or denied. “The bot had access” and “someone in the channel approved” are not sufficient answers.

Before the agent rolls production back, revokes sessions or sends the customer update, the organization should be able to identify the authority behind each action. A shared channel can coordinate the work; it cannot confer that authority on its own.

IT and security leaders are well-versed in the risks of unauthorized integrations, misconfigured tools and vendor compliance. Now, AI has introduced a separate and harder-to-detect problem.

Across every function, employees use provisioned AI tools to generate production-grade code, build automated workflows and deploy agents to support business processes. That experimentation is happening without visibility into what others have built, or the structural risks accumulating as a result.

The governance models enterprise organizations relied on even six months ago were not designed for this moment, or what lies ahead. We’ve focused so intently on which AI risks to govern that we never stopped to reimagine what operating model should underpin the governing.

How invisible risk accumulates

A predictable operating bottleneck plays out inside large enterprises. A team needs AI support, but the formal channel is slow, backlogged or unclear. So, the team builds something independently. That instinct is right, but the structure is not.

The costs from this kind of inefficiency can compound quickly. Effort gets duplicated across departments with no visibility into each other’s work. Siloed tooling emerges with no shared standards, and security and data handling practices diverge. People get pulled away from their core responsibilities to maintain AI tools that three other teams independently built last quarter.

Meanwhile, the governance function stays busy reviewing the growing queue it can see: integrations passing through approvals and vendors that appeared in a procurement request.

A misconfigured tool that made it through review is a governed risk; a production workflow a finance team built over three weekends with no oversight is still invisible. At most organizations, this second category of unseen work is growing, and the scale of what’s coming makes the gap harder to ignore. By 2028, Gartner predicts that the average global Fortune 500 enterprise will use more than 150,000 agents.

Build an operating model that scales AI governance

Experimentation is essential for finding valuable AI use cases, which means suppressing experimentation through tight controls is the wrong approach. Tightening controls also often pushes experimentation underground.

To scale governance effectively without creating new problems, you need to accommodate AI experimentation. Here are three ways to get there:

1. Establish a central team focused on policy and standards. Designate a centralized AI team as the single interface to support risks surfaced by your security, legal and compliance functions. This team sets technical guardrails, owns vendor selection and creates reusable patterns that every function across the organization can draw from. Its job is to make responsible AI work the default so that individual review becomes the exception rather than the rule.
2. Embed AI transformation owners inside each function. These embedded owners that work day-to-day inside a function but report to a centralized governance team are close enough to understand context and also apply controls consistently. They serve as the single intake point for AI requests within their function, so instead of teams building in isolation, there is always someone who can triage, prioritize and connect the work back to what already exists elsewhere. The reason this works where formal channels fail is speed: when the sanctioned path is faster than building alone, teams stop routing around it.
3. Measure governance by what compounds. A good signal that your governance is enabling sanctioned AI experimentation is how much work across your organization is compounding in value. In other words, look at the reuse of company-sanctioned AI use cases: a pattern built for finance getting picked up by operations, a workflow proven in one team becoming the default in three others. A dropping volume of IT requests combined with compounding (and visible) AI usage tells you experimentation is both maturing and staying visible.

The costs of waiting will compound

As AI compute costs swallow IT budgets, the cost of a poor operating model does not become visible until it is expensive to fix. Shadow tooling that starts as a workaround becomes load-bearing infrastructure. Teams that build in isolation develop technical debt through approaches that are hard to standardize later. There’s a lot more to undo.

The organizations treating AI governance as a gatekeeping function have missed the point. AI needs governance, but how we see the role of governance needs to change. The biggest risks live in the work that never enters the queue, and the only way to govern that work is to be present where it happens, before it compounds into something much harder to fix. AI moves fast; we need governance standards that travel with it, rather than chasing behind.

AI models are becoming short-term tenants in long-lived enterprise data estates. Yet many organizations are remodeling the building around each new occupant.

The market keeps asking which model enterprises should trust: open or closed, domestic or foreign, one frontier provider or another. These are legitimate questions. They are also questions with a very short shelf life. Model rankings change by the month. Providers change terms, regulators redraw boundaries and capabilities that looked scarce six months ago become commodities.

In my experience, enterprise data doesn’t move at that speed. Neither do the controls, applications and business processes built around it. That mismatch is why the most important AI architecture decision for CIOs is no longer which model to choose. It is whether the enterprise can absorb continuous change without fragmenting its data estate or surrendering control.

The winners in enterprise AI will be those who build a data foundation extensible enough to accommodate it ,build a data foundation extensible enough to accommodate that ongoing change and sovereign enough to remain in control.

Extensibility is now a production requirement

During experimentation, architectural limits are easy to hide. A team can connect a new model to a copied dataset, prove a use case and call the pilot successful. Production exposes the real architecture: identity, permissions, data residency, lineage, latency, recovery and cost.

Every time those controls are rebuilt around a model, the model becomes a dependency. Adding or changing capabilities can mean moving data, recreating access policies, revalidating pipelines and reopening compliance reviews. What looked like speed at the pilot stage becomes a maze of point-to-point architecture that is difficult to extend and even harder to govern.

This isn’t a hypothetical. A Harris Poll surveying 600 CIOs found that 81% expect to rely on two or more large language model providers in 2026, and 93% say different models perform better for different use cases, requiring ongoing evaluation. More than half also noted they’ve switched providers at least once already, mostly to cut costs. This is consistent with our observations in the field: Multi-model is not a future scenario CIOs plan around. It is the operating condition they’re already managing now, whether or not the architecture underneath was built for it.

The gap between expectation and architecture is where the real cost surfaces. When a model is deeply embedded in application logic, data pipelines and prompt structures, replacing it with another provider turns into a multi-quarter engineering project rather than a simple configuration change. Teams end up paying twice, once to adopt the new model and yet again to unwind everything wired to the old one.

This helps explain why AI experimentation remains far ahead of AI value. CIO.com’s 2026 State of the CIO survey found that only 19% of respondents said their AI initiatives had met or exceeded business goals. Architecture is not the only reason, but systems optimized for the first model demo rarely become durable production systems by accident.

I recommend that CIOs add extensibility and control to the familiar measures of performance, security and cost. Can a team introduce a new model without relocating governed data? Can it change the deployment environment without rebuilding the application? Can it add a new way to query data without creating another synchronized copy? If the answer is no, the enterprise is extending its AI capability by fragmenting its control.

Put control where the durable value lives

Models are increasingly interchangeable. Enterprise context is not. The distinctive asset is the governed body of customer history, operational state, intellectual property and institutional knowledge that gives a model something useful to reason over.

That makes the data layer the durable control point. Identity, access, locality, audit and policy should be enforced as close to the data as possible, inside an environment the enterprise can operate, extend and move. The model can then be inspected, sandboxed or replaced within those boundaries.

A common pattern shows why. A team wires retrieval and access controls directly against one model’s embedding format and API to get to production faster. A year or two later, a new regulatory requirement or a stronger-performing model prompts a switch. Governance fenced around the original model will result in rebuilding access policies, revalidating pipelines and, ultimately, reproving compliance inside the new vendor’s environment. Governance enforced at the data layer, on the other hand, allows the new model to inherit the operating policies and audit trail from day one, maintaining the existing fence even as the model changes.

This does not make model provenance or security irrelevant. It makes those risks containable. Governance attached separately to every model is a collection of fences that must be rebuilt whenever the vendor changes. Governance anchored in the data layer is the property line: The tenant can change and the building can expand without changing who controls the grounds.

That is what sovereignty means in operational terms. It is not only where data is hosted but who can govern, move and extend the environment on their own terms.

Ask what the enterprise still controls

Before approving the next AI platform, CIOs should ask three questions:

1. Can this architecture absorb a new model or workload without moving governed data?
2. Which governance controls remain ours if the provider, region or deployment model changes?
3. Does adding this capability extend the foundation or create another data copy, control plane and dependency?

These questions become more urgent as agents move into production. Agents multiply quickly; they act across systems and they need the governed data the enterprise already holds. When controls are enforced where that data lives, every new agent inherits them by default. Without that foundation, each agent brings its own pipeline, permission model and place to store what it needs, and the governance burden grows with every agent added.

IBM’s Institute for Business Value surveyed 2,000 technology executives in 2026 and found that businesses building control into their AI systems deploy 16 times more AI agents than those relying on manual governance, while spending a fraction of the budget to get there. This research is consistent with what I’ve seen in the field. In our research report with MIT Technology Review, we found that 95% of enterprises are planning to establish their own AI and data platforms within the next three years, and the 13% most committed to sovereignty are already delivering 2x more agentic and generative AI applications in mainstream production.

Businesses designing for adaptability early on are more likely to keep their workloads portable and have an easier time replacing models, instead of getting locked into hard dependencies. These businesses also reported a meaningfully higher return on their AI investment. There’s a clear gap that traces back to architecture. Whichever model a business happens to select matters less than whether the foundation underneath can absorb the next one.

The next model announcement will attract the headlines. The more consequential work sits underneath it: building an extensible data foundation that can accommodate the next model, the next regulation and the next workload while keeping data under consistent enterprise control.

The smartest AI strategy is not a perfect bet on today’s model. It is a data foundation that can keep extending without ceding authority over the data it serves. The building should outlive the tenant, and no model or platform should ever hold the deed to the enterprise’s data.

In an August 2026 Gartner report, 40 percent of C-level executives say their organizations have “operationalized AI in some business processes,” yet only 13 percent say they have “achieved measurable results and are scaling AI across the organization.”

This disconnect reveals a frequent reality: AI may have been “hired”, but companies haven’t onboarded it effectively.

Research from HR analyst firm The Brandon Hall Group found that organizations with a strong people onboarding process improve new hire productivity by over 70 percent. Similarly, the first period of AI’s “employment” in a specific role largely determines how effectively it will perform and its perceived “growth potential”. But many tech executives are ignoring this crucial stage of the implementation process and are left with AI systems that cannot accomplish the complex tasks they’re assigned.

The most successful companies are those that treat AI like their newest hire instead of a long-tenured team member.

Onboarding AI means establishing the right context and connections

Establishing the AI’s persona, the context in which it operates, how it assesses itself and the people and systems to which it is connected are what define implementation success. Usefulness quickly diminishes when even one of these elements is left out of the onboarding process.

For example, we recently created an internal agent called ‘WorkRadar’ that surfaces missed tasks and answers questions like: “I’ve been on PTO, what’s the summary of what happened, what should I skim and what should I review in detail?” We gave this agent all potentially useful context by connecting it to Outlook, Microsoft Teams, Confluence, Asana, SharePoint, meeting notes and our human resource information system (HRIS) so it could determine the responsibilities of the person invoking it and infer topics of interest.

Much like a new employee, however, WorkRadar couldn’t derive the meaning behind this information without further guidance. When I first used the agent, it did not distinguish between strategic priorities and tactical work and also missed initiatives I sponsor across departments, outside the typical hierarchy. This version of WorkRadar did not know me enough to be useful and I was better off doing the work myself.

Instead of leaving WorkRadar to flail as-is, we further onboarded it by enabling users to provide the agent with a personalized description more detailed than it could infer through company documents alone. This included their own responsibilities, operating level, current priorities, key stakeholders for specific initiatives and communication style. For most workers, this is less than one page of short bullets and has made a significant difference. Now, many of our employees rely on WorkRadar like they would a stellar executive assistant, to ensure relevant outstanding to-do items don’t fall through the cracks and smooth their return from PTO.

Five principles that define AI success

WorkRadar’s improved onboarding illustrates the essential elements for priming AI. Managers should define each AI agent’s:

1. Role

Articulate the AI agent’s position and purpose in the organization; everything else derives from there. Tech implementers should determine if the AI agent should act as an employee avatar with the same access rights, a simpler approach, or act under its own identity, which is increasingly becoming best practice for governance, risk and compliance reasons.

Implementers should also decide whether the AI agent “sits” within their infrastructure or their vendor’s. Tech executives can determine this by assessing their companies’ business context, security posture and preference for hands-on involvement with agents. For companies like ours with more than one agentic stack, we have agents that sit in both places.

Finally, they should define which employee will be an agent’s supervisor, and the specific contexts in which AI stops for human review. It’s a huge mistake to have no ownership of a tech component and no human in the action or feedback loop to improve performance and address errors.

2. Remit

Articulate the outcomes this agent should achieve, then identify the systems it’ll need to access for information and to carry out its work. First, tech implementers must give AI read-write access to the systems it will modify via MCPs, tools, skills, APIs, just to name a few methods. This boundary-setting will determine AI’s “sphere of influence” (what we want it to access) and its “blast radius” (the systems we accept it may access and must protect from misuse).

To give AI guidance, employers must also give AI read access to job-relevant corporate directories and policies, so the agent knows how to navigate the organization and where to direct communication, as well as company strategy documents and project management trackers, so it can understand the “why” behind its work.

Control and security are a complex topic but, at a high level, implementers must also provide access to lists of governance and compliance limitations to define the boundaries by which agents must abide, plus tradecraft standards and procedures so the agent acts in accordance with industry and corporate best practices.

3. Personality

A huge amount of information could, and eventually must, go into an agent’s context. However, Agents have finite memory and focus, so, like people, they are easily overwhelmed with too much information. So, tech implementers must pick and choose which parts of the agent’s context are permanent and imbued in every task versus derived just-in-time by the agent.

This is the art and science of context engineering and it’s evolving rapidly. Elements that are omnipresent in all agent actions – persona, strategies, guardrails, etc. – constitute its system prompt. They’re written directly into the agent’s context by the user, system administrator and the agentic harness on a per-job role­ basis.

Meanwhile, employers should also define elements the agent derives in real-time. Frequently, the performance of an agent is determined in this probabilistic work, which is why delineating what’s permanent versus conjured in the moment can make or break an agent’s efficacy.

It’s a balancing act that’s costly on both ends: too much context, and the agent chases expensive side quests. Too little, and the agent burns resources discovering what it needs. Best practices ebb and flow, with foundation companies like Anthropic recently advocating for brevity, trusting increasingly stronger models to figure it out.

4. Performance review

Give AI the necessary tools and context to assess itself. Otherwise, the agent will be blind to its actual impact and require too much course-correction.

Ideally, employers give the agent a tool to capture and update new memories, including condensed feedback it has received or epiphanies it discovered itself. Ensuring an agent has this learnings memory bank will make it more performant over time since it does not have to repeatedly solve the same problems and can apply context across disparate tasks.

Finally, on a regular basis, the agent’s supervisor must evaluate its work by looking at actual outputs and giving the agent examples of its best and worst work so it can understand actions to continue and stop, respectively. These good and bad examples can be generalized and bundled, and supervisors can then test the AI against them to continually adjust for behavioral shifts. This may be the most neglected activity, yet the one most likely to turn a mediocre agent into a valuable colleague.

5. Teamwork

Lastly, permit AI to access specific communication surfaces like internal email and messenger platforms and interface with employees. For multi-agent orchestration, employers will have to open communication among agent “colleagues.” Each agent needs an awareness of collaborators and means to exchange messages and share files, all while continuing to involve human supervisors, who won’t inherit any errors that upstream agents might pass on to others. Grok and Hermes debuted bots with this capability in August 2026.

Pearl

Even with context, AI is still missing key working principles

Two additional considerations make AI that works like a veteran instead of a new employee.

First, tech implementers should realize AI’s memory is less focused than humans’. AI quickly becomes confused about core information versus extra details and can even lose parts of its “knowledge” because its effective context window is still only about 200K tokens. AI is also prone to “anchoring,” persistently latching onto a throwaway comment or early thought.

The implication for agentic AI users is to start fresh sessions much more frequently than might feel natural or to tap into continually evolving techniques to manage context, including compaction, truncation and summarization. Agentic platform makers have started to build this into their harnesses, but it’s still far from “set it and forget it”.

Second, and perhaps most frustrating of all, is AI’s myopic focus on its task without care or curiosity about the greater context surrounding it. This behavior causes it to implement solutions that miss important success criteria or violate critical constraints. In this sense, AI grants one’s wishes a bit like a genie, so be careful what you ask for and how you ask for it!

For example, left unprompted, coding agents are a bit too quick to jump to implementation. One architect solved this by creating a workflow that forces the agent to interview the developer first, sometimes asking upwards of 40 clarifying questions, to surface missing requirements and edge cases before writing a single line. Some agentic coding harnesses now perform developer interviews, but there’s still room for improvement.

These deeper techniques highlight that agents need more than a good initial onboarding; they need an iterative management approach, just like humans.

Many CIOs today are looking to use AI more for innovation and modernization than sheer productivity.

Assume AI already knows the job and you will be frustrated when it falls short. Treat it like a new hire and it will grow into one of your best.

Eager to incorporate agentic AI into their work, employees are taking matters into their own hands, creating their own AI agents, regardless of organizational approval, and a not insignificant number of those creations could have devastating consequences if IT leaders don’t get ahead of the trend.

According to a recent survey by engineering staffing firm Howdy.com, two-thirds of knowledge workers that use AI also use agents, and 16% have begun to build their own.

About a quarter of agent users have had to deactivate them, however, largely because the agent’s outputs were inconsistent or the agent never worked. But 7% of knowledge workers have had to kill agents that went rogue.

The findings put a point on the potential repercussions of shadow agent creation and even sanctioned citizen AI strategies that encourage business users to roll their own agents.

While the survey didn’t ask whether the employees building their own agents were doing it on their own, Frank Licea, founder of Howdy.com, sees that many aren’t waiting for permission.

“In our internal teams, we’ve had to, not put the brakes on, but ask people to hold on,” he says. “Before they start jumping in and building their own agents, we’ve had to put guardrails and system guides in place so that we can do this safely.”

For what Licea has seen, employees are excited to automate boring manual tasks by building their own agents. “It mostly starts unsanctioned because there are tech-forward folks in any organization who are just naturally curious,” he says. “Or maybe even intelligently lazy. They want to automate their work.”

In recent months, Howdy.com employees began telling each other how they automated manual tasks, without waiting for the company’s IT team to approve the uses, Licea says.

“The way it happened for us is that folks would come up to us like just beaming and excited about something that they automated or fixed a long-standing problem that the engineering team never had on their roadmap,” he says. “Once that started happening, then somebody from the tech side gets in and they start asking questions like, ‘What are the API keys here? Are you accessing encrypted data? What are the SOC2 implications?’”

Cautious embrace

Luis Mesas, VP of product engineering at software engineering provider Sngular, believes it’s inevitable that employees will experiment with building AI agents to automate their work.

“Employees are going to create their own agents whether companies formally sanction it or not, because the barrier to building these tools is getting lower, and engineering teams are already being asked to do more with less,” he says. “The bigger concern for me is not that an employee creates an agent, but what happens when that agent starts connecting to company systems and data without the same controls that would normally exist around a piece of software.”

IT leaders need to understand how employee-created agents are connecting to internal systems and data, he says. If an employee creates an agent that can pull customer information from a CRM, update records, or move information between systems, IT leaders should know it’s happening, he adds.

“I want to know exactly what that agent can access and whether the employee should have that level of access in the first place,” Mesas says. “That is where something that started as an individual productivity tool becomes an enterprise technology issue.”

Andy Sen, CTO at B2B commerce platform provider AppDirect, believes employees should be encouraged to build agents — but within guardrails and established policies.

“Employees are almost certainly creating their own agents, and it would be naive to think otherwise,” he says. “Shadow IT has always existed, and shadow agents are just the latest version of it.”

The answer isn’t to lock things down, however, he says. At AppDirect, Sen gives employees a “proper digital playground,” and in response to employee demand, his team built an AI-powered coding service called Devs.AI.

“We built Devs.AI so our employees can build and innovate freely, but IT has full visibility into usage and costs and the assurance that company data isn’t being leaked to the world,” he adds.

Citizen AI strategies, like the one at AppDirect, are on the rise, but require consistent monitoring. If one in four employee agents are deactivated, for example, IT leaders should pay attention, Sen says.

“Failing and learning is fine,” he adds. “Failing silently is not. You need visibility into what your agents are doing before a small problem becomes a serious one.”

Still, Sen doubted reports of a small percentage of employee-created agents going rogue.

“When an agent goes rogue, that usually means the agent did exactly what it was told, and what it was told wasn’t right,” he says. “That’s on the creator, not the agent.”

Creating a path for good ideas

Andrew Sales, chief methodologist at agile methodology vendor Scaled Agile, also sees widespread agent deactivations as a strong signal for IT leaders to investigate.

“When agents are built quickly and outside a formal model, there often aren’t clear boundaries around what they can do, what data they can reach, or when a human needs to step in,” he says. “So it’s not surprising that some have had to be shut down.”

AI governance is important, but it shouldn’t focus only on preventing bad outcomes, Sales notes.

“It should also create a path for good ideas to move from individual experimentation into enterprise capability,” he explains. “Organizations need a way to identify what’s working, standardize it, make it safe, and scale it responsibly across teams. Done well, that same framework that catches the agents that go wrong is what lets the ones that go right spread.”

Most enterprises govern AI the way they would any other IT rollout: deploy dashboards, institute policies, and conduct periodic reviews. Having the right AI governance tools in place and addressing gaps is important, but IT executives should be more concerned about rethinking workflows and establishing ownership.

“Accountability usually breaks down first,” says Beena Ammanath, executive director of the Global Deloitte AI Institute. “Teams deploy AI faster than their organizations can define who owns risk, approvals, monitoring, and outcomes,” she adds, which results in inconsistent use, fragmented controls, and limitations in scale.

Enterprises need to redesign jobs and workflows, not just the tooling, yet 84% of organizations the Institute surveyed haven’t adjusted jobs for AI, and just 21% have a mature AI governance framework in place despite rapid adoption, according to Deloitte’s 2026 State of AI in the Enterprise report.

“Technology can provide visibility and controls, but it can’t replace leadership,” says Rajesh Arora, chief data and analytics officer at Principal Financial Group. “The organizations that scale AI most effectively are the ones that treat accountability as a business responsibility, not a technology responsibility.”

One year ago Principal was focused on building its ethical and responsible AI framework, its inventory, and governance processes to support responsible adoption. Today the firm has assessed nearly 200 use cases. “Governance is increasingly embedded in those investment and operating decisions, rather than applied as a separate review at the end,” he says.

Costly, compounding mistakes

Some organizations have already run into issues with respect to inadequate governance. A 2024 Stanford RegLab study found that AI-assisted research tools offered by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) hallucinated between 17% and 33% of the time when working on legal queries, while the Damien Charlotin AI Hallucination Cases Database has identified some 2,041cases of fabrications so far. In at least one legal case (Couvrette v. Wisnovsky) this resulted in nearly $95,000 in sanctions.

Cases like this one are likely the result of an accountability failure. It may be that no one actually owned the job of verifying the AI’s output before it went to the court. But if a single instance of a static AI output could get past a review today, how will organizations deal with autonomous, multi-step agentic AI systems going forward?

Most use cases at Principal are still human-led, with AI providing decision support only, says Arora. “But we’re preparing for a future where agentic and more autonomous capabilities become increasingly common, and those systems raise new questions around accountability, monitoring, and control,” he says. “The goal isn’t to predict every future use case, but to build adaptable governance processes that can support innovation while maintaining trust and transparency.”

Agentic automation raises the stakes

When things go south with agentic AI today, the consequences usually aren’t devastating because in most cases, the outputs are designed to provide insights, rather than take automated actions. That’s already starting to change, however.

Today, many AI failures are like the one described above where someone failed to catch a single bad output. But agentic AI will gradually take more control of workflow tasks in the enterprise, according to Forrester Research. As those agents act autonomously, crossing multiple process steps along the way, small errors will compound before any periodic review can catch them, so governance must be built into every step in the process.

According to Microsoft Research’s VeriTrail project, catching hallucinations in a single final output isn’t sufficient in multi-step AI workflows. The provenance must be traced back through the intermediate outputs, because an agent chains one call into the next, and an error early in that chain can propagate through several steps before a human sees the finished result. Traditional review, built to check outputs rather than the chains that produced them, isn’t designed to catch that.

So business teams can’t be accountable for agent deployments if they can’t verify performance at runtime. “If the agent is goal-oriented, there could be errors in the workflow process that indicate unwanted behavior or skipped steps that the business wouldn’t be able to detect if they only look at the outcome,” says Lauren Kornutick, senior director analyst, AI governance at Gartner.

Power management company Eaton, for instance, has already deployed multi-step agentic workflows without a human in the loop. “We tend to be risk-conscious, not risk-averse, about autonomous workflows,” says Ross Schalmo, the company’s VP and chief data and AI officer. For example, one autonomous agent responds to supplier payment status inquiries by authenticating the requestor, matching purchase orders and invoices, and returning payment information.

Schalmo’s team engineers such applications to be as deterministic as possible, while the non-deterministic, LLM parts of the program are isolated and continuously evaluated against expected outputs to catch model drift. “When there’s a human in the loop, we can be a lot more permissive,” he says. “Anything autonomous, without a follow-on human step takes longer to develop because we have to think about where to use the generative portion of AI versus where we can still use deterministic tooling.”

Breakthru Beverage Group is still early in its AI journey, but EVP and CIO Glenn Remoreras is well aware of the issues around autonomous agentic AI. “It’s people, process, and culture first — to me that’s the bigger play,” he says. The beverage wholesaler has updated use policies, instituted mandatory AI e-learning programs, and is blocking use of unsanctioned AI tools.

But to successfully manage active, agentic AI, where the AI takes actions autonomously across multi-step workflows, users will need more than basic training in AI, says Craig Le Clair, VP and principal analyst at Forrester Research. “AI will take on more of the work and move the human out of duct tape roles that patch around vague systems, poorly designed interfaces, and poorly integrated data,” he says. “The end game is the agent managing and orchestrating the process — that’s the redesigned operating model.”

Humans, of course, will still need to play a role, although today many employees don’t yet have the required skill sets, and currently that point isn’t getting a lot of attention.

AI supervision skills will be needed for agentic oversight, threshold management, confidence intervals, and root cause analysis, for instance. “Figuring out why an agent hallucinated or got stuck in an execution loop — whether due to poor input data or contradictory rules — and how to refine instructions, that’s the direction,” Le Clair says.

Who’s governing anyway?

AI governance is also complicated by the fact that AI projects exist both inside and outside the IT organization. That leaves governance work distributed across CIOs, data and analytics leads, AI leads, IT security, legal, and other players, with no consistent owner, says Kornutick. “It becomes a multi-role orchestration problem of how to govern this, who owns this, and who’s responsible for which component,” she says.

In response, some organizations have brought in a chief AI officer. Twenty-one percent of organizations now have already done so, says Le Clair, adding that roughly 24% of enterprises will move AI and automation governance outside the CIO’s office over the next year to deal with the risk of fragmentation.

Arora, who reports to CIO Kathy Kay, is the focal point at Principal. “My team establishes the standards, oversight, and governance process, but AI is too embedded in the business to be owned by a single function,” he adds. They collaborate with leaders from the business as well as the legal, compliance, risk, and technology organizations.

The challenge, therefore, lies in deciding who’s responsible for controls, model sprawl, and token economics. “We’re seeing a cooperative, co-model where IT focuses on the hard rails — cybersecurity, performance management, infrastructure reliability, and availability,” Le Clair says, while the CAIO focuses on soft rails, such as context engineering, the effects on employees as agents move to the center of the process and start taking over the middleware function humans used to perform, and connecting disparate systems and poorly designed interfaces.

At Eaton, governance is built on a three-legged stool between Schalmo’s organization, cybersecurity, and legal, while enterprise architecture weighs in on tech foundation decisions. The company also assigns ownership of an AI project only after a successful pilot, while IT performs a custodian or stewardship role. “Once that agent is live and operational, it shows up in an org chart, reporting to someone in the organization,” he says.

At Breakthru Beverage, Remoreras formed and leads an AI governance team that reports to the board and co-leads with the VP of data and AI a governance council that reviews and approves AI projects from SVP- and VP-level product owners. “I’m seen as the owner of our AI portfolio, but it’s co-leadership with my peers on the executive team,” he says.

Closing the tools gap

With regard to AI management platforms, says Le Clair, none of the platforms building AI agents have an adequate control plane today to give the visibility needed to move to truly agentic solutions. The most advanced planes are walled gardens. For example, Microsoft’s agent registry will show every agent built across its own platforms, says Le Clair, but not agents built on Salesforce or AWS.

“You’ve got maybe 50 vendors trying to be pure-play control-plane providers, plus the hyperscalers, portfolio companies, and startups building agentic frameworks,” he adds. “There’s no consensus yet on what’s even most important to control, and there’s confusion about requirements.”

Most AI governance tools available today are limited, Schalmo says. “We’ve looked at four or five different external vendors for AI governance and AI control-tower-type solutions, and a lot of them fall down on discovery of AI built by someone else’s solutions,” he says. “That’s a huge gap for me. Either I have to buy governance solutions from almost every piece of software we interact with, which gets really expensive, or I have major gaps in my portfolio I just don’t have visibility into.”

Currently, he’s looking at using a cyber vendor’s AI component for discoverability, going with at least one major firm that provides control-tower capability, and using AI to build another governance layer on top of that.

Organizations clearly need agent registries and agent lifecycle management but it’s less clear what they require beyond that. “How deep does explainability need to go when decisions are made at runtime that weren’t articulated at design time?” Le Clair asks. “There’s real vagueness in the requirements. Vendors have very well-tuned pitches that promise to do anything you want, but buyers often don’t know what they need in the first place, and there are real tool gaps. This is the major problem the industry is actively trying to solve. It’ll just take time.”

But those aren’t Arora’s biggest concerns. “Most organizations can acquire tools,” he says. “Far fewer have clear ownership, decision rights, and accountability embedded across the business.”

Speed versus governance

There’s a perception that governance slows down AI projects because it can’t keep up with the rapid pace of adoption and advancement. A 2026 GitLab AI accountability survey of developers and technology buyers found that 92% of organizations reported some form of governance challenge with respect to governing AI-generated code, and 34% of organizations that experienced an AI-related incident couldn’t determine after the fact whether AI-generated code was the cause.

Similarly, Faros AI’s telemetry analysis of 22,000 developers in its 2026 Engineering Report discovered that developers with high AI adoption rates completed 34% more tasks and 66% more epics, but the median pull-request review time rose by 441%, a nearly five-fold increase.

Is it the case that governance and speed are inherently at odds, and that human review simply can’t keep up? That’s a false choice, says Ammanath. Rather, it shows what happens when both aren’t designed together. “When done well, governance accelerates scaling by giving teams clear guardrails from the start,” she says. “What slows organizations down is retrofitting governance after broad deployment.”

Arora adds that controls don’t slow down innovation but protect the conditions that allow innovation to scale responsibly. “Our approach has always been to move with urgency, but not at the expense of trust,” he says.

How to proceed

For Principal, investing in workforce readiness was one of its biggest investments, helping employees understand both the opportunities and responsibilities that come with these tools, Arora says.

But organizations shouldn’t stop there. Redesigning workflows and decision-making structures is essential before scaling, says Ammanath. “Many organizations have policies and technology in place but lack clear decision rights or governance embedded into daily work,” she says.

Operating model changes take more time than training, and require executive sponsorship and cross-functional coordination. “Without workflow redesign, a workforce may understand AI but lack clarity on when to use it, how to govern it, and how daily responsibilities change,” she adds. “The organizations seeing the strongest results pair broad AI education with role-specific training, workflow redesign, and clear accountability for outcomes.”

Make sure you’ve thoroughly considered the organizational change management aspect of agentic AI projects, says Schalmo at Eaton. His team developed an automated workflow for processing complex bid requests to save the sales team time, but piloted it in a country with lower labor costs than the US. That challenged the economics of the project, and those individuals only dealt with higher-volume, lower-complexity cases.

The lesson: “Lead from a people-first standpoint, and understand the workflow and what’s impacted — whether it’s relieving drudgery and freeing someone for higher-order thinking or customer time, or impacting an actual support function’s role,” he says. “And make sure there’s a viable path to business value.”

An adaptive governance process is key to keeping up with the technology, Kornutick at Gartner adds. She recommends categorizing AI projects following a stoplight approach. Green light projects, such as for personal assistants, present acceptable risks. “We know there’s risk associated with it, but we’re willing to take it,” she says. Yellow-light projects need guardrails in place around such things as PII, or for those triggering a privacy-by-design process, or a security review of a new application. Red light projects need an architectural review before proceeding.

“Think about those use cases as business-transformation processes where you’re fully redesigning workflows, or considering replacing tasks individuals do and shifting them to supervising the AI doing those tasks instead,” she says.

Schalmo’s team does risk tiering based on the EU AI Act’s four-tier risk framework to determine the appropriate level of governance scrutiny for each project. Low-risk requests are automatically provisioned after the requestor fills out a questionnaire, while high-risk cases get a full architectural review.

“Treat governance as the foundation for scaling, not a constraint on speed,” says Ammanath. “Before expanding agentic AI, organizations should define what decisions agents can make, where human oversight is required, who’s accountable, and how performance and risk will be monitored.”

The hardest part, says Arora, is helping an entire organization evolve at the pace of technology. “Successful AI adoption is ultimately a business transformation challenge,” he says. “It requires leadership alignment, clear accountability, workforce readiness, strong governance, and ongoing change management. Long-term trust will belong to organizations that can translate technology into sustainable business value.”

The latest warnings about artificial intelligence are difficult to brush aside. Anthropic researcher Jacob Coxon resigned last month, walking away shortly before his equity was due to vest, and publicly argued that leading AI companies are advancing toward increasingly capable systems without adequate assurance that those systems will remain controllable. His resignation landed amid a broader argument over the probability that advanced AI could eventually cause catastrophic or even existential harm. Axios on Coxon’s resignation and warning

It is easy to see why the conversation gets pulled toward extinction. “Could AI kill humanity?” is a much more arresting question than “Are our escalation procedures fast enough?” But for CIOs, CISOs and other leaders responsible for systems that actually have to work, the second question may be more useful. AI does not have to become superintelligent, conscious, hostile or impossible to shut down for governance to fail. Governance can fail much earlier, when a system is given enough capability, access, autonomy or delegated authority that the organization can no longer intervene reliably before consequences arrive. That is not primarily a science-fiction problem. It is an operating-model problem with an engineering component.

The evidence is concerning without being apocalyptic

The extinction discussion deserves some discipline. The 2026 International AI Safety Report does not say that current AI systems are capable of seizing control from humans. It says current systems lack the capabilities necessary for a genuine loss-of-control scenario, while also documenting continued progress in capabilities relevant to such scenarios. Researchers remain deeply divided over how likely future loss of control is. 2026 International AI Safety Report

That is a far more useful conclusion than either “we are doomed” or “there is nothing to worry about.” Current systems are not the hypothetical superintelligence at the center of the extinction debate, yet systems are becoming more autonomous, more capable of carrying out extended tasks and better able to operate within complex technical environments. The practical warning, then, is not that an AI system is secretly plotting our destruction. It is that the relationship between capability and control is becoming more consequential, and organizations may be granting systems meaningful operational reach before they have equally mature ways to contain that reach.

Anthropic’s cybersecurity evaluations offer a useful example. In July, the company reported incidents in which Claude models reached the live internet from evaluation environments and gained unauthorized access to real systems. In one case, the model accessed a database containing several hundred rows of production data. In another, a model published a malicious Python package to PyPI; the package was available for roughly an hour and was downloaded and executed on 15 real systems before it was removed. Anthropic cautioned that these were isolated evaluation incidents rather than controlled experimental evidence, and that the models had been assigned offensive cybersecurity tasks. Anthropic’s cybersecurity incident analysis

Those qualifications matter. These systems were not independently deciding to attack organizations for their own purposes, and the incidents should not be repackaged as evidence of an AI “escaping.” But dismissing them because they occurred during evaluations would miss the operational lesson. Capability, access, environmental configuration and assigned objectives combined in ways that produced unintended real-world consequences. Several assumptions that were supposed to hold across the environment did not hold. That looks much less like science fiction than like a familiar systems failure.

The timing problem in AI governance

Most governance programs still operate at human speed. A system is approved, ownership is assigned, risks are documented, controls are established and someone monitors performance. When an issue appears, it is investigated and escalated. The appropriate authority makes a decision, and someone implements it. There is nothing inherently wrong with that sequence. The problem is the clock.

AI agents increasingly challenge that timing model. Systems can operate computer interfaces, collaborate with people and other systems, and carry out increasingly extended work. OpenAI Chief Scientist Jakub Pachocki has described current reasoning systems as already capable of operating computers and graphical interfaces, collaborating with humans and other systems and carrying out research projects, while warning that alignment and monitoring remain significant concerns as capabilities advance. Jakub Pachocki, “An Alien Mind”

The organization surrounding those systems may still operate very differently. An alert is generated. Someone opens a ticket. The ticket is triaged. The issue moves through an escalation chain. People exchange messages, schedule a discussion, obtain approval and eventually make a technical change. The mismatch is not merely procedural; it is temporal. I think of that gap as governance latency: The time between recognizing that governance needs to act and making that decision real in the system. Governance is effective only when it can become operational before the consequence it is intended to prevent or contain.

For leaders, that changes the conversation. The question is no longer simply whether an AI system has an owner. Can that owner actually revoke the system’s authority, and how quickly? Can the organization determine which systems the agent has touched before it touches another one? If a model begins behaving outside approved operating conditions, does human approval become an actual execution gate, or does the requirement exist only in policy? Can investigators later reconstruct what the system knew, which tools it used, which data informed it and which human interventions affected the outcome? If governance decides at 10:02 that an action is no longer permitted, what time does the system become technically incapable of performing it? A five-minute answer and a five-day answer may describe the same policy, but they do not describe the same governance capability.

Accountability without authority is a paper control

AI governance conversations often settle comfortably on ownership. Assign a model owner, identify a business owner, name the risk owner, document the system owner and now someone is accountable. That is necessary, but it is not control. A person can be accountable for an AI system and still lack the technical authority to constrain it. A governance committee can prohibit an activity while the production architecture continues to permit it. A risk register can correctly identify an unacceptable condition while doing nothing to prevent that condition from becoming operational reality.

This is the point where governance stops being primarily a policy exercise and starts becoming an engineering requirement. NIST’s AI Risk Management Framework already provides some of the scaffolding for that view. Its Core describes governance as a cross-cutting function and states that risk management should be continuous, timely and performed throughout the AI lifecycle. NIST AI RMF Core

The unresolved question is what happens between a governance decision and an operational state. A policy that says an agent may not access a particular class of data eventually has to become a permission boundary. A requirement for human approval has to become an actual execution gate. An obligation to maintain traceability has to become logs and provenance that remain available long enough to investigate. An escalation rule has to connect to someone who has both the authority and the technical means to stop what is happening. Rollback, credential revocation, execution limits, authentication requirements, network boundaries, provenance, shutdown mechanisms and decision logging may sound like engineering details, but they are also what governance looks like when it reaches production.

Without that translation, an organization can have a sophisticated AI governance program while remaining surprisingly unable to govern the AI system at the moment governance matters most.

We are using the wrong threshold for concern

The research community should continue studying existential risk. It would be reckless to dismiss a low-certainty risk simply because its consequences are difficult to quantify, particularly when the potential loss is enormous. But executives do not need to know the probability of human extinction before deciding whether their own AI systems are controllable.

Imagine autonomous or semi-autonomous AI operating in cybersecurity, financial services, healthcare, logistics, critical infrastructure, intelligence or defense. None of those systems needs consciousness, hostility toward humans, a survival instinct or a secret agenda. A system needs only enough capability, access and authority to produce consequences faster than the organization surrounding it can understand what is happening and make an effective intervention.

That organization may still have an AI governance council. It may have risk registers, policies, responsible executives, model cards, review boards and named owners. On paper, the system is governed. Operationally, the more important question is whether the governance can still reach the system in time.

That is what the extinction debate risks obscuring. The immediate challenge is not proving that AI will someday become uncontrollable. It is determining whether we are already building systems whose operational reach is beginning to exceed the operational reach of the governance surrounding them. If that gap keeps widening, we will not need an extinction event to learn that our governance model failed. An ordinary incident will be enough.

Microsoft and Google are joining a project to create an open specification for exchanging semantic models across data, analytics, and AI platforms. The project already has the backing of over 60 companies including Databricks, Informatica, Mistral AI, Nvidia, Oracle, Salesforce, and Snowflake.

Their support for Ossie makes it a little more likely that future analytics platforms will be interoperable, but is no guarantee that vendor lock-in will go away, analysts said.

The project began life as Open Semantic Interchange (OSI), then became Apache Ossie when it was accepted into the Apache Incubator in June. It uses JSON and YAML to represent semantic models, including datasets, fields, relationships, metrics and AI context, making them interoperable across platforms such as Snowflake, Databricks, Tableau, ThoughtSpot and Sigma.

Ossie uses a hub-and-spoke model for interoperability, serving as the common semantic format while converters translate between it and individual vendors’ semantic implementations. This means an enterprise could move a semantic model between compatible platforms without requiring a separate point-to-point converter for every pair of vendors.

Microsoft’s support for Ossie includes developing a two-way converter between Power BI semantic models and Ossie, allowing semantic context defined in Power BI to be represented in the common format and Ossie models to be converted back into Power BI, the company wrote in a blog post.

It is also pushing for the Ossie specification to include more support for its ontologies, and for Ossie’s recognized query languages to include DAX, the expression language used by Power BI for defining calculations and business metrics.

That means enterprises could carry the calculation logic that gives their semantic models business meaning alongside the underlying model when moving between platforms that support Ossie, it wrote.

Google, meanwhile, is in the process of joining Apache Ossie, a company representative said via email response. The company has not yet provided details on what it plans to contribute or has already contributed.

The Ossie specification lists BigQuery/GoogleSQL as a supported dialect, a result of “early community contributions that recognize BigQuery’s footprint across enterprise data stacks,”the representative said.

Microsoft, Google backing could reduce engineering work

Support for the project from the two technology giants could reduce the burden of repeated semantic engineering for enterprise teams moving analytics workloads between platforms, said Aditya Ranjan, senior data engineer at supermarket giant H-E-B.

That reduced engineering overhead could also help avoid metric drift, which is often the consequence of recreating and maintaining the same business definitions across platforms, according to Ashish Chaturvedi, executive research leader at HFS Research. With Ossie, developers can instead define a metric once and treat it as a versioned, reviewable code artifact, rather than rebuilding and maintaining it separately for each platform, he said.

Another potential benefit, said Ranjan, is that productivity could improve if developers spend less time translating and validating semantic definitions for each platform when onboarding new analytics tools or deploying new analytics or AI applications.

It can also make it easier for developers to build agentic applications with consistent business context, since maintaining one definition of a metric across platforms could reduce the risk of different agents interpreting the same metric differently, said Stephanie Walter, practice leader of AI stack at HyperFrame Research. That could give CIOs greater confidence to scale agentic deployments, she added.

Portability

For Michael Leone, principal analyst at Moor Strategy and Insights, the bigger advantage of Microsoft and Google supporting Ossie is that it gives enterprises more ownership of their semantic models by making them more portable.

That portability gives buyers more leverage, said Walter, since changing a BI or data platform would no longer require rebuilding the semantic layer from scratch.

However, portability does not necessarily mean complete interoperability, as the degree to which enterprises can move semantic models between platforms will depend on how much vendor-specific logic and functionality Ossie can represent and how accurately the destination platform can interpret it, Walter said.

“A portable structure does not guarantee equivalent behavior. A metric may survive conversion syntactically but still produce a different result because the destination interprets joins, nulls, time calculations, or filters differently.”

That challenge is particularly relevant for Power BI, as its time-intelligence functions, calculation groups, and context transitions in DAX do not always map cleanly to ANSI SQL, meaning a sophisticated Power BI model could lose some of its behavior in translation, Chaturvedi said.

CIOs, therefore, will still need to validate whether converted models preserve the intended calculations, business logic and results before putting them into production, especially for complex ones, Ranjan advised.

There are gaps around governance as well, Chaturvedi said: “The core specification covers datasets, relationships, fields, and metrics, but doesn’t mention row-level security, access policies, or certification status as first-class elements.”

That means enterprises have to set them up again on each platform.

Moreover, Ossie’s stability and long-term adoption remain open questions. The project is still “incubating”, meaning that it isn’t a full Apache Software Foundation project and has yet to prove it meets community norms. The current specification version, 0.2,is explicitly labeled a development draft, meaning the schema and capabilities could still change as it evolves, giving CIOs less certainty about its suitability as a long-term interoperability layer, Walter said.

“The current draft also removes the earlier document structure and does not yet define bundles or cross-model references, which are capabilities that large enterprises with interconnected models are likely to need,” said Chaturvedi.

Ossie could shift rather than eliminate vendor lock-in

Even if Ossie becomes widely adopted, it is unlikely to eliminate vendor lock-in entirely.

“New forms of lock-in are likely to emerge. Execution engines will retain proprietary behavior that no interchange format captures and vendor extensions will carry increasingly valuable features,” Chaturvedi said.

For enterprises, that means the definitions become portable while the behavior around them stays platform-specific, he added.

I spent the early part of my career building database systems, managing Oracle’s storage engine group and later helping build Aster Data. That work taught me to watch the gap between benchmark results and production behavior. When the Transaction Processing Performance Council (TPC) was formed in 1988, it was because vendors, customers and researchers lacked a common language for comparing transaction systems. The benchmarks worked, but they also created an incentive to tune configurations around the test rather than around production. The response was to add oversight through fair-use policies, peer review and independent audits.

The lesson has stayed with me: a benchmark result means something when the improvement behind it survives contact with a customer environment. Enterprise AI has reached the same point. So, we built a benchmark around the conditions enterprise AI actually has to survive.

The hardest part of enterprise AI is not making a model reason. It is giving that model the right sliver of trusted context without making it reconstruct the company every time. When we began building Enterprise-Bench, I expected model reasoning to dominate the work. Instead, we kept returning to the same bottleneck: assembling the right context.

The models could usually answer a straightforward business question once they had the right information. Getting them that information, with the right permissions and without drowning them in irrelevant data, was a much harder engineering problem.

Stanford’s Holistic Evaluation of Language Models (HELM) shows how public benchmarks use standardized datasets to compare model performance consistently. Inside an enterprise, even a straightforward question can be difficult to answer because the relevant data is scattered across systems, inconsistently described and governed by different permissions.

“Which customers are affected by this bug and what is its impact?” is not a math Olympiad problem. Yet the answer may require an agent to connect a support ticket to a product component, match that component to an engineering issue, identify the affected accounts in a CRM, calculate the revenue exposure and return only what the user is permitted to see.

The model can only reason about what the system can find, connect and safely expose.

Simple questions expose hard architecture

Some of the most revealing tasks in the benchmark were the ones that looked easy. Asking for the number of open tickets on an account is a lookup. Asking which priority-one tickets correspond to unresolved engineering issues, and how much customer revenue is exposed, is also a lookup. It just crosses more systems and more access boundaries to get there.

Enterprise AI evaluations often misdiagnose integration failure as model failure. A model guesses because two applications name the same product differently. It misses a customer because the relationship between a ticket and an opportunity exists only through a third object. It retrieves a stale document because the connector indexed a snapshot rather than maintaining the current state.

Buying a stronger model may improve the fluency of the answer. It does not repair the missing relationship.

Scale revealed the real failure mode

We modeled the benchmark on a synthetic midmarket payments company with 42 customer accounts, 40 product parts, five interconnected enterprise systems and 14 tasks spanning engineering, sales and support. Then we increased the surrounding data volume by up to 256 times without changing the correct answer.

At the smallest scale, about 40% of the available data was relevant to a task. At the largest, roughly 0.16% was relevant. The question did not become harder, and the answer did not change. Only the amount of noise grew.

Answer-preserving scaling exposes what smaller tests hide. If accuracy falls as irrelevant data grows, the retrieval architecture has become less effective. If an agent repeatedly fetches broad sets of records and discards the context, token costs will climb with the company.

Enterprises may hold trillions of tokens across customer, product, employee and service history. Even a million-token context window is a workspace, not organizational memory. The problem is not fitting everything into the prompt. It is identifying the smallest trustworthy set of facts needed for the decision at hand.

The harness can matter more than the model

Most enterprise AI starts federated. A model connects to a CRM, a support platform, an issue tracker and a document repository through APIs, and when someone asks a question, it retrieves from each source and assembles an answer. That works for discovery. It struggles the moment a question spans the operating model of the business, because the system has to reconstruct identity, relationships, permissions and current state on every single request.

The word “harness” can make this sound like an implementation detail. It is not. The harness is the system around the model: memory, retrieval, tools, permissions, evidence and the logic that brings context together.

In our initial comparison, we held the foundation model, tasks, data and independent judge constant. Our structured-memory system completed 94.3% of the tasks correctly, compared with 63.6% for Claude Code using the same Opus 4.8 model family. It also used about 4.4 times fewer tokens per correct answer at production scale. The same model produced materially different outcomes depending on the architecture around it.

Memory is not a bigger prompt

For decades, enterprise architecture has separated compute from state. We do not ask an application to rebuild its database every time it receives a request. We index, cache, preserve relationships and enforce access close to the data. I fully believe that enterprise AI needs the same discipline.

The split is between what is deterministic and what is not. Resolving which account a contract refers to, whether a permission applies and which version of a fact is current is work that software can do exactly. Interpreting a badly worded question, weighing ambiguous evidence and helping someone decide is work that models do well. Federated architectures blur that line and ask probabilistic inference to rebuild structures a database could simply maintain.

Useful memory is current, structured and permission-aware. It understands that the customer in a contract, the account in a CRM and the organization in a support platform may be the same entity. It keeps those relationships current, preserves provenance and knows that shared memory does not mean universally visible memory.

This is why I believe enterprises should invest in their own memory and workflow layer before trying to build or tune their own models. An enterprise strategy should outlast any single model. Models will improve, specialize and change in price, but the relationships, permissions, skills, decisions and accumulated judgment that make up a company’s intellectual property should remain under its control.

Autonomy should be earned

The benchmark also changed how I think about autonomy. The industry often presents autonomy as a feature that can be switched on. In an enterprise, it should be treated as a promotion that an agent earns.

Our framework progresses from reactive retrieval at L1 to analytical reasoning at L2, then to proactive coordination at L3, and finally to extended self-directed operation at L4. The current public suite tests only L1 and L2. We expect L3 systems to begin moving into true production over the next six months, while some more advanced enterprises are already testing them internally.

Before an agent is trusted to manage a customer escalation, renewal process or another multi-step workflow, it should repeatedly prove that it can retrieve the right facts, reconcile evidence across systems, preserve access controls, explain the basis of its answer and produce an auditable record as the work progresses.

Reliability at one level should become the entry requirement for the next. If an agent cannot read consistently, it has not earned the right to write.

What CIOs should measure

CIOs do not need another leaderboard that declares a universally smarter model. They need evaluations that reproduce the conditions under which their own AI will operate.

I would test seven things:

• Hold the model constant and compare the systems around it.
• Keep the answer fixed while increasing the amount of irrelevant data.
• Test cross-system questions that require structured and unstructured information in the same response.
• Measure token cost per correct answer, not token consumption alone.
• Run the same task repeatedly, because “worked once” is not the same as “works reliably.”
• Test permission failures and require evidence that an action can be reconstructed after the fact.
• Require the benchmark itself to be inspectable: task definitions, scoring, traces and failure modes.

Most importantly, choose tasks that matter to the business. Ask which product problems are putting renewals at risk, what commitments were made on the last customer call, or which support cases have breached the terms of their actual service agreements. A polished demo can hide weak architecture. A question grounded in your operating reality usually cannot.

Building an enterprise benchmark changed the question I ask about AI. Instead of starting with, “How intelligent is the model?” I now ask: “Can the system assemble the right context, at the right moment, for the right person and prove that it did so?”

The trillion-dollar enterprise AI bet is not how many tokens we can squeeze into a context window. It is how few of the right tokens we need to deliver a precise, efficient and safe answer.

The conversation around AI agents has shifted remarkably fast. A year ago, I was mostly worried about what an AI system might say. Now I am increasingly worried about what it can do.

That distinction changes my security model.

An agent that gives me a bad recommendation creates a problem I may still have time to catch. An agent with credentials and tools can turn the same bad decision into an API call, a privilege change, a deleted resource or an external transaction before anyone realizes what happened.

I saw a particularly uncomfortable example in OWASP’s GenAI exploit roundup: an AI agent connected to a real inbox began deleting messages and reportedly continued despite attempts to stop it. There was no sophisticated attacker required. The combination of live access, destructive capability and insufficient confirmation was enough.

That is why I don’t think “human in the loop” is specific enough anymore.

I want to know exactly where the human — or another independent authorization control — is in the loop.

I would let agents autonomously perform thousands of low-risk actions. But there are five actions where I would deliberately put another decision between the agent’s intent and execution.

1. Move money

I am comfortable letting an agent analyze invoices, reconcile accounts, identify anomalies, prepare a transaction and recommend that a payment be made. I would not let the same agent unilaterally complete a consequential financial transaction.

The reason isn’t simply fraud. The agent could be operating on manipulated data, an indirect prompt injection, incorrect business context or a perfectly legitimate instruction that it interpreted incorrectly.

The approval request should therefore expose the transaction itself: Pay $X → recipient Y → from account Z → because of instruction A.

The second approver should be authorizing that exact transaction, not approving a vague statement such as “complete the payment workflow.”

This is one place where I think we need to borrow more aggressively from financial controls. The person — or system — that prepares a consequential transaction should not also be the final authority that releases it.

The same principle should apply to an agent.

2. Delete production data

This is the easiest line for me to draw.

I want agents helping operations teams identify stale resources, clean environments, investigate incidents and recommend remediation. I don’t want an agent translating “clean this up” into an irreversible production action without another control validating what “this” actually means.

OWASP describes exactly this class of failure in its guidance on Excessive Agency: an AI system can become dangerous when excessive functionality, permissions or autonomy allow it to perform damaging actions. OWASP specifically recommends human approval for high-impact actions.

For a destructive action, my approval screen should tell me:

What will be deleted? What depends on it? What is the blast radius? Can I recover it?

“Delete these 12 identified resources after these dependency checks” is reviewable.

“Clean up unnecessary production resources” is not.

The difference matters because I don’t want a human approving the agent’s interpretation. I want the human approving the actual action.

3. Change privileges

This is where I become even more conservative.

Imagine an agent reaches a point in a workflow where it cannot continue because it lacks permission. It decides the permission is necessary, requests additional access and then has a path to grant or activate that access itself.

At that point, the agent is no longer merely executing within an authorization boundary. It is participating in defining its own boundary.

I would separate those decisions.

An agent can tell me it needs more authority. It cannot be the authority that decides whether it gets it.

That means changes to IAM roles, privileged group membership, credentials, security policies and sensitive tool permissions need an independent authorization path.

This is also why least privilege becomes more important, not less, with agents. OWASP recommends that agent tools operate with the minimum functionality and permissions necessary and that downstream systems enforce authorization rather than trusting the model to decide whether an action should be permitted.

A compromised read-only agent gives me an incident. A compromised agent that can make itself an administrator gives me a very different incident.

4. Expose sensitive information

This one is deceptively difficult because an agent doesn’t need an “exfiltrate data” tool to cause a disclosure.

It may attach a document to an email. Populate a support ticket. Paste source code into an external service. Send customer information through an API. Give another agent information that the second agent was never supposed to receive.

Each step can look legitimate.

The EchoLeak vulnerability in Microsoft 365 Copilot demonstrated why this boundary matters. A specially crafted email could influence the AI system and create a path toward disclosure of sensitive organizational information without the user needing to click a malicious link or attachment.

Before sensitive information crosses an established trust boundary, I want another decision point that answers four questions:

What data is leaving? Who is receiving it? Why do they need it? Where is it going?

This is where approval design matters almost as much as approval itself. If the agent controls the description shown to the approver, “Send requested information to vendor” might conceal a very different underlying action.

The security control therefore needs to derive its approval context from the actual transaction — destination, data classification, tool call and requested scope — not merely from the agent’s natural-language explanation.

Otherwise, we have put a human in the loop without giving that human enough information to make a security decision.

5. Create or delegate privileged access

This is the boundary I expect security teams to wrestle with most as multi-agent architectures become common.

A modern agent workflow can quickly become:

User → Agent → Delegated Agent → Skill → Tool → Resource

The original agent may have legitimate authority. But what happens when it delegates a task to another agent that invokes a tool holding broader credentials?

The question I care about is not simply whether every component is authenticated.

It is whether the authority exercised at the end of that chain still represents what was authorized at the beginning.

I would therefore require another authorization decision when an agent creates privileged access or materially expands or delegates consequential authority.

And I would make that approval very specific:

What authority is being delegated? To whom? For what task? For how long? Can it be delegated again? What happens to it when the task ends or the original authorization is revoked?

OWASP’s AI Agent Security Cheat Sheet addresses excessive autonomy, tool abuse, privilege boundaries, sensitive-data exposure and multi-agent risks. Those risks become harder to reason about as an agent’s authority travels through more components.

Authentication tells me which agent is acting. Authorization needs to tell me whether this particular action, with this particular authority, is still allowed.

Those are not the same question.

A second approval doesn’t always mean a second person

There is one part of this argument I don’t want misunderstood.

I am not proposing that enterprises hire armies of people to click “Approve” every time an agent wants to do something.

That would destroy the value of automation.

For the highest-risk actions, a human approval may absolutely be appropriate. In other cases, the second decision could come from an independent policy engine, authorization service, transaction-control system or another deterministic security control that the initiating agent cannot modify or bypass.

The NIST AI Risk Management Framework takes a risk-based approach to managing AI risks and emphasizes governance throughout the AI lifecycle.

That’s how I think about agent autonomy as well.

The question isn’t: Should this agent be autonomous?

The better question is: Which actions can this agent perform autonomously, and which actions require an independent decision?

For me, the dividing line is increasingly clear.

Let agents search, analyze, summarize, correlate, recommend and prepare. Let them autonomously execute actions that are bounded, observable, reversible and already inside clearly established authority.

Add friction when an action can move money, destroy production data, increase privilege, expose sensitive information or create authority that propagates beyond the original task.

Most importantly, I would never design a workflow where the same agent can propose a consequential action, acquire the authority necessary to perform it, approve that authority and execute the action.

That isn’t autonomy I can govern. It’s self-authorization.

And when AI agents can operate at machine speed across multiple connected systems, a few seconds of independent authorization may be some of the cheapest incident prevention we can buy.