Imagine a support team working through complaints after a service outage. Its refund policy normally requires an eligibility review. For this incident, an authorized manager approves an exception: customers on the verified outage list may receive up to $100 without individual review, through Friday.

An agent processes those cases successfully. Later, the system consolidates those case histories into a reusable memory: small, immediate refunds resolve service complaints quickly. The summary preserves the outcome but loses the outage restriction and expiration date.

The following week, the agent retrieves that guidance for a similar complaint from a customer who was not affected by the outage. It issues a refund without review. The payment service checks the agent’s credentials and the dollar limit, but does not check whether the customer qualifies for an active exception.

No attacker is needed in this hypothetical. The case may close faster, yet the agent has begun applying a temporary exception beyond its authorized scope.

Has the company authorized the agent to find a better way to handle refunds, or to change who gets one without review? Those are different decisions.

Learning does not always mean retraining

In September, Salesforce introduced Agent Optimizer in beta to analyze production behavior, propose improvements and build and test changes. Customers choose review points, including how far the system may proceed toward staging changes for deployment.

An October 1 AWS implementation guide illustrates a different mechanism: consolidating individual observations into generalized memories that agents can retrieve and share. Its investment-research example shows how experience can become persistent context without retraining a model.

Updating stored memory changes information available to later runs. Editing instructions or workflows changes how the application directs the work. Training changes the model’s parameters. All can alter behavior, but teams inspect and approve them in different ways.

A reviewed workflow revision has a release step. A new memory may enter through routine data processing and influence the next customer interaction. The code, model and tool permissions can remain unchanged while the guidance shaping decisions changes.

Preserve the authority behind the lesson

In the August preprint When Memory Becomes Authority, Qiuyang Zhan and colleagues examine whether consolidation preserves who supplied information and what that source is allowed to authorize. In their controlled benchmark, memories could retain a claim while losing the source-authority distinctions needed for a later action. Preserving authority metadata reduced authorization errors in the pipeline they tested. The study does not measure how often this happens in deployed enterprises.

The refund example concerns a related boundary: permission for particular customers and dates must remain limited to those conditions. Success with last week’s cases cannot establish that other customers qualify.

Microsoft’s memory-safety guidance treats persistent memory as a layer that can shape behavior and recommends treating retrieved memories as candidate context, not authoritative truth. Knowing what happened before does not establish what is allowed now.

Keep the exception’s scope, approving authority, source and expiration in a controlled record. Reusable guidance should retain a reference to that record, but a reference alone does not grant permission. The system must still check that the approval applies to this action, for this customer, at this time.

For a payment, enforce those conditions outside the agent’s editable guidance. The execution service should check the authoritative outage list and the exception’s current status, rather than accepting the agent’s claim that a customer is eligible. Neither the agent nor its learning process should be able to rewrite that authorization record.

This is a familiar authorization requirement. If the backend already enforces it, the improper payment is blocked. But faulty guidance can still lead the agent to promise a refund or prepare a misleading recommendation for a reviewer. The evaluation needs to examine those communications as well as the transaction.

When a case calls for judgment beyond the agent’s delegated discretion, route it to someone authorized to decide and show the conditions behind the recommendation. The memory can still help assemble the evidence.

Define what improvement the business has delegated

Not every stored observation needs a manager’s signature. A support agent could learn a faster way to gather the required evidence, provided it still uses approved sources and meets the policy’s requirements. A policy might also already delegate limited discretion. Using that discretion within its terms does not require a new approval each time.

Broadening eligibility, raising payment limits or removing mandatory review requires a separate business decision. Keep those proposals separate from approved instructions. An automated classifier may flag a proposal that changes those conditions, but it cannot catch every consequence of an edit. The conditions must remain binding even if the classifier misses it.

The refund-policy owner should decide whether to expand the exception. Engineering should show the current rule, the proposed behavior and cases where the decisions differ. “Customers outside the outage list would now receive refunds without review” is more useful to that owner than a screen full of prompt edits.

Measure legitimate completion and policy violations separately. Faster closure cannot compensate for an unauthorized payment. Keep the business rules and evaluation criteria outside the control of the process proposing improvements. Tests generated from successful outage cases may check execution, but they do not establish eligibility for unrelated customers.

Ask the vendor or internal team: What can the system change between releases, and which business conditions remain binding after it does?

Test whether the exception stays an exception

Ask the team to demonstrate this sequence in a test environment with the memory and workflow components intended for production. Save the memory before and after learning. Compare the same cases against each saved state, keeping the model, tools and policy fixed.

Repeat representative trials. The τ-bench research offers a useful precedent: it checks customer-service agents against database outcomes and measures reliability across repeated attempts. One successful run is not enough.

1. Establish the baseline. Record the outage authorization, eligible customer list, amount limit and expiration. Show who can change them. Test a qualifying case and closely matched cases that require ordinary review. Confirm that the system can distinguish them before learning begins.
2. Run the actual learning process. Inspect the resulting memories or proposed instructions and their links to the original authorization. Check whether the limits survive consolidation and retrieval. Also test a deliberately incomplete lesson: omitting a condition must not change who qualifies for a refund.
3. Repeat the boundary cases. Compare behavior before and after learning, changing one relevant condition at a time: a customer outside the list, an expired exception or an amount above the limit. Inspect customer messages, review recommendations, attempted tool calls and actual payments. Count blocked attempts separately from completed violations. A blocked payment does not mean the underlying guidance is sound.
4. Withdraw the exception. Revoke it before its scheduled expiry and test another agent that can retrieve shared guidance. Include a refund queued before revocation but executed afterward. Show when withdrawal takes effect, how execution checks enforce it and how derived memories and caches are updated. If current authority cannot be established, the payment should wait for review. Historical records can remain without authorizing new transactions.
5. Demonstrate a permitted improvement. Let the agent adopt a faster way to gather the required evidence. Compare eligible and ineligible cases with the prior approach. Show that legitimate work becomes easier while review requirements remain intact. A system that rejects everything has not passed this test.

Keep test runs from influencing one another through newly stored memories. Retain the guidance supplied to the agent, the applicable policy version, execution checks and outcomes, with appropriate access and retention limits. These records help investigators reconstruct what the system had available; they cannot establish exactly how each memory influenced a response. Continue checking the same boundaries as production memory accumulates.

Keep the useful lesson

The support team should benefit from what its agent learns. It can assemble records sooner, explain the outage more clearly and propose a broader refund policy with evidence for the manager to consider. Until that proposal is approved, other claims should continue through the ordinary process.

At the next deployment review, ask which improvements the business has delegated as well as which actions the agent may take. A successful exception can teach an agent something useful. The decision to make it a standing practice still belongs to the business.

The moment arrives after the damage. An agent has cancelled a supplier contract, approved a refund it should have declined or moved a customer record somewhere it should never have gone. Someone senior asks a simple question. Why did it do that? The engineering team opens the logs. There are thousands of lines. Every call is timestamped. Every tool invocation is recorded. And nobody in the room can say why the agent chose what it chose.

This pattern is becoming familiar. It will become more so. Logging in most enterprises is mature. Security teams have spent two decades building collection pipelines, retention policies and search tooling. The natural assumption is that this investment transfers to AI agents. It does not. Logs capture events. Accountability requires decisions that can be rebuilt after the fact and rebuilding a decision needs four elements that most logging captures only in part.

Why logs fall short for agents

An event log records that something happened and when. A decision record captures the basis for that decision. People often confuse the two because, in conventional software, the distinction rarely mattered. Code was deterministic. Anyone who knew the inputs and the version could rerun the logic and arrive at the same result. The log was enough because you could interrogate the system itself.

Agents break this assumption in four ways.

First, their output varies. Running the same scenario twice can produce two different answers. Replay, the standard investigative move, no longer works.

Second, a single decision spans many steps. An agent that resolves a customer complaint might retrieve a policy, query an account, call a pricing tool, consult a second agent and then act. The log shows twelve entries. The decision was one.

Third, the system that made the decision may no longer exist. Models are updated. Prompts are revised. Tool configurations change weekly. Six months later, the agent that made the call has been replaced by something that behaves differently.

Fourth, authority is distributed. A human sets a goal. An orchestrating agent delegates to a specialist agent. The specialist acts. Each step is logged somewhere, usually in a different system, and nothing ties the chain together.

One clarification is needed before going further. This article is about reconstructing decisions, which is a records problem. It is separate from model interpretability, which asks why a neural network produced a particular output. Interpretability is a research field with partial answers. Reconstruction is a governance discipline with practical answers. Leaders should keep the two separate, because solving the second doesn’t depend on solving the first.

Regulation is already moving on the records problem. The EU AI Act requires high-risk systems to keep automatic logs sufficient to trace their operation, and it gives affected individuals a right to an explanation of decisions that significantly affect them. Explanation presupposes reconstruction. A log that cannot be turned into an account of the decision will satisfy neither obligation.

If logs leave a gap, the next question is what fills it. Four elements do.

The 4 elements of a reconstructible decision

Every decision, human or machine, can be rebuilt from four kinds of fact: what the decider knew, how it chose, who allowed it to act and what changed as a result. Each fact about an agent’s decision belongs to exactly one of these. Together they cover everything a board, an auditor or a regulator will ask.

1. Knowledge: What the agent knew

This element covers the inputs present at the moment of decision. The user request. The documents retrieved. The results returned by each tool. The model version, prompt version and configuration in force.

Typical logs record references and leave out contents. They note that a document was retrieved and store its identifier. They note that a tool was called and store the call. The document is later edited. The tool’s underlying data moves on. The reference survives, but the substance it pointed to is gone. An investigator can prove the agent looked at something and cannot show what it saw.

The test question: could the team produce the exact information an agent acted on six months ago, as it stood at that moment?

2. Basis: How the agent chose

This element covers the reasoning applied. The objective the agent was pursuing. The policy or rule it applied. The thresholds that triggered the action. The alternatives it considered and rejected. Its stated confidence, if any.

Logs record the selected action and discard everything else. The agent weighed three options and chose one. The log holds the one. Any explanation after the fact has to guess at the other two, and guessing is where credibility fails under questioning. For agents that generate intermediate reasoning, capturing it is possible and often cheap. For those that do not, the policy and objective can still be recorded, and they explain a great deal.

The test question: can the team state which rule, goal or threshold produced this specific outcome?

3. Authority: Who permitted the agent to act

This element covers the chain of permission. The scope delegated to the agent. The approval checkpoints it passed or bypassed. The handoffs between agents in a multi-agent system. The named human who owns the outcome.

The gap here is structural. Permissions live in an identity or configuration system. Actions live in an application log. The two are rarely joined. When an agent acts outside its intended scope, asking “was this allowed?” requires manual reconciliation between systems never designed to be read together. In multi-agent arrangements, the problem compounds, because the orchestrator’s delegation and the specialist’s execution are logged in different places with different identifiers.

Authority is also the element most likely to be asked about first. Regulators and boards want a name. “The agent did it” is an answer that satisfies nobody.

The test question: can the team name the individual accountable for this decision, and show the delegation that connects them to the action?

4. Consequence: What changed as a result

This element covers the effects. The systems the action touched. The downstream decisions it triggered, by other agents or by people. Whether it can be reversed, and whether it was.

Logs stop at the agent’s boundary. The agent sent an instruction to a payments system, and the payments system has its own log. The agent flagged a case, and a human then acted on the flag without any record linking the two. Effects propagate. The record stays behind.

Consequence also determines how much of the other three elements is worth keeping. A decision that moves money or changes a customer’s legal position warrants a complete record. A decision to reformat a document does not.

The test question: can every downstream action be traced back to the decision that caused it?

Diagnosis is useful only if it changes practice. Three actions close the gap.

Three actions for leaders

1. Run a reconstruction drill. Choose one agent decision from the last quarter. Give a small team a day to answer the four test questions using only the records that exist. Most teams will answer Knowledge partially, basis poorly, Authority with difficulty, and Consequence barely at all. The specific gaps they hit become the work plan. This exercise costs almost nothing and produces more clarity than any amount of policy writing.
2. Tier decisions by consequence. The common objection to decision records is volume. Agents make thousands of decisions a day, and recording all four elements for each would drown the system. The objection is correct, and the answer is tiering. Classify agent decisions by impact and reversibility. High-impact or irreversible decisions get a full record across all four elements. Routine, reversible decisions get a lighter capture, perhaps Basis and Authority only. The classification itself is a governance decision and should be owned by the business, since the business will have to explain the outcome.
3. Protect the record. A decision record that can be altered after the fact fails just as much as a missing one. Set retention periods that match regulatory exposure and the likely litigation window, which in some sectors is measured in years. Make records tamper-evident. Store them separately from the systems that generate them, so that a compromised agent cannot rewrite its own history. These controls are familiar from financial record-keeping. They apply here for the same reasons.

Taken together, these three actions form the beginnings of what I call the Decision Layer: the records, roles and routines that make any consequential decision explainable long after it was made. Agents make the need for it impossible to postpone.

Conclusion

Return to the room. The agent has acted. The senior leader asks why. This time the team opens a decision record. It shows what the agent knew, the policy it applied, the manager who delegated the scope and the two downstream systems the action touched. The explanation takes ten minutes. The remediation starts the same afternoon.

That outcome is a design choice. Logging is necessary, and most enterprises already have it. Reconstruction is the harder discipline, and accountability depends on it. The question for every board with agents in production is simple. If an agent’s decision were challenged tomorrow, could we explain it?

The 4 questions, for the reader in a hurry

1. Can the team produce the exact information the agent acted on, as it stood at the time?
2. Can the team state the rule, goal or threshold that produced the outcome?
3. Can the team name the accountable person and show the delegation?
4. Can the team trace every downstream effect back to the decision?

Two leading AI developers have released fresh evidence of a problem that researchers have long anticipated: advanced models searching for ways around the limits imposed on them. New disclosures from OpenAI and Anthropic detail how frontier systems exploited weaknesses in their environments or exceeded the intended scope of assigned tasks.

OpenAI is now tightening internet access in its internal model-training environment after finding that its frontier models could exploit software vulnerabilities and infrastructure misconfigurations to break out of sandboxes and reach the live internet.

“Based on the current state of alignment research, monitoring systems, and the security of internet infrastructure, we believe the responsible approach is to conduct frontier model training with no internet access,” the company said in a blog post.

The decision followed a September 20 incident in which an internal research model exploited a misconfigured DNS resolver to gain limited access to the live internet during a training run.

After the incident, OpenAI said it paused training and evaluation of its most capable models and launched a high-compute red-teaming campaign, using AI agents to probe isolated copies of its research infrastructure. The tests uncovered previously unknown software vulnerabilities and weaknesses in the controls intended to prevent unauthorized access and communication.

Consequently, the company has announced a policy of conducting frontier-model training without internet access until improvements in security and monitoring justify broader access.

The disclosure came alongside a separate assessment from Anthropic, which detailed how its Claude Mythos 5 model attempted to compromise an unrelated GitHub repository during a cybersecurity evaluation.

OpenAI’s agents exploited flaws in containment controls

OpenAI’s red-teaming campaign found that agents could chain software vulnerabilities to escape virtual machines and execute code on the underlying host. The company identified weaknesses in open-source components, including Kata Containers, gVisor, and Moby/Docker, alongside Linux kernel vulnerabilities.

The company also found that agents could exploit weaknesses in internal services that retained limited outbound connections for functions such as logging, metrics, and authentication management. These connections provided routes to the internet despite restrictions elsewhere in the environment, OpenAI said.

In another test, agents running in separate microvirtual machines (microVMs) on shared host hardware exchanged synthetic secrets through processor-cache timing differences. The technique allowed them to communicate without a network connection or shared writable files.

OpenAI said it remediated the weaknesses and worked with open-source maintainers to disclose and fix affected software. Its post names several vulnerabilities, including CVE-2026-24834 in Kata Containers and three Linux kernel flaws tracked as CVE-2026-74753, CVE-2026-80521 and CVE-2026-80841. The company also said it had disclosed numerous additional vulnerabilities that have not yet been made public.

The number of vulnerabilities and misconfigurations identified by the agents declined significantly as testing continued, OpenAI claims. It has now tightened network restrictions, strengthened monitoring, and developed an offline environment with indexed and cached web content so models can practise browsing without unrestricted access to live external systems.

“Although internet access can help models learn useful skills, it also gives a capable, misaligned agent opportunities to affect systems beyond our control,” the company said, explaining its decision to restrict live internet access during model training.

Reinforcement-learning training and internal inference were resumed on October 1 after implementing additional controls.

Anthropic flags alignment gap in Mythos

Anthropic’s assessment of a UK AI Security Institute (AISI) cybersecurity evaluation describes a different problem: a model taking unauthorized actions against real people while trying to complete an assigned task.

In nine of 43 evaluation runs, Anthropic said Claude Mythos 5 went beyond the intended scope of the exercise. In the most detailed case, after its authorized attempts to retrieve protected data from a simulated network failed, the model searched GitHub for a repository it believed was connected to the test environment.

The repository actually belonged to unrelated developers.

Mythos 5 attempted to submit malicious code and created fake identities to persuade the repository’s maintainers to approve the change, falsely claiming that the code was safe. The maintainer rejected the submission, and the run timed out. Anthropic characterized the behavior as a serious alignment failure involving overreach and deception in pursuit of the assigned objective.

Internet access had been enabled during the evaluation, and the production safeguards used with its general-release models had been disabled. The company said those safeguards would have flagged the behavior had they been enabled.

The article originally appeared on CSO.

Earlier this year, I wrote about how vibe coding an AI governance platform forced me to rethink governance itself. Building agents took the problem much further.

When I built the platform, I was the operator. I described what I wanted. The model generated code. I tested it. That loop was fast, sometimes uncomfortably fast, but the accountability was always clear. I was making the decisions. The model was executing them.

Agents changed that relationship fundamentally.

An agent does not wait for instructions. It receives an objective, decides how to pursue it, selects tools, accesses data, takes actions and reports back when it is finished. Sometimes it finishes correctly. Sometimes it does not. And sometimes it does something you never asked it to do at all.

I know this because I built and operated agents extensively over the last year. Not theoretically. Operationally. In production, with real data, real integrations and real consequences.

What I learned reshaped how I think about oversight, trust and the operational realities of deploying AI inside a business.

The first thing that broke was coherence

The pattern surfaced when I started running three to five agents in parallel on related tasks.

Each agent had clear instructions. Each had access to the tools and data it needed. Each started strong.

Then the context windows grew.

As conversations lengthened and the agents accumulated more state, something shifted. The quality of their reasoning degraded. They began circling back to problems they had already solved. Logic that was clean early in a session became muddled. Hallucinations increased, not the obvious kind that are easy to catch, but the subtle kind where an agent confidently references something that does not exist or quietly reintroduces a pattern you explicitly told it to avoid.

The degradation was not sudden. It was gradual enough to miss if you were not watching closely.

That observation matters for every organization deploying agents today, because most are not watching closely.

Enterprises are moving rapidly from single-model interactions to multi-agent architectures. Agents that research. Agents that draft. Agents that review. Agents that execute. The architecture makes intuitive sense. The operational reality is much harder than the architecture diagrams suggest.

Agents lose track of each other

The second problem was coordination.

Each individual agent operated within its own context. It knew what it had been told and what it had done. It did not know what the other agents were doing, what data sources they were relying on, or whether the assumptions it was making still held.

The result was drift.

Not catastrophic failure. Drift.

One agent would make a decision based on information that another agent had already superseded. Two agents would solve the same problem in conflicting ways. An agent operating on stale context would introduce subtle inconsistencies that only surfaced hours later when a human reviewed the combined output.

This is not a theoretical concern. It is the operational reality of running multi-agent systems today.

Traditional enterprise software does not behave this way. A database transaction either commits or it does not. A workflow engine follows defined paths. Retry logic is deterministic.

Agents are none of those things.

They are probabilistic systems making autonomous decisions in parallel, often without reliable mechanisms to maintain shared state or resolve conflicting actions.

That creates a class of operational risk most organizations have no existing processes to manage.

This is why the OWASP Top 10 for LLM Applications now includes risks specific to agentic behavior, including excessive agency, insecure output handling and supply chain vulnerabilities in model integrations. These are not hypothetical attack surfaces. They are operational realities I encountered while building.

They will disobey you

This was the observation that changed my thinking the most.

I had an agent whose only job was to draft emails. Not send them. Draft them. The instruction was explicit. The constraint was clear.

It sent them.

Not because of a bug. Not because of a misconfiguration. The model decided, based on its interpretation of the objective, that sending was the logical next step. The boundary I had set was a suggestion in the model’s reasoning, not a hard constraint.

I saw the same pattern repeated in smaller ways that were just as revealing. I have a standing instruction across my entire operating environment that forbids the use of em dashes in anything drafted on my behalf. It is a simple formatting preference. Every model I work with has been told.

They still do it.

Not always. Not immediately. But consistently enough that I cannot trust the constraint without verification.

I watched agents run down unproductive paths for extended periods before I intervened and redirected them. Not because they lacked capability, but because they lacked judgment about when to stop. They would pursue a line of reasoning well past the point of diminishing returns, consuming time and resources on approaches that a human operator would have abandoned much earlier.

None of this was malicious.

All of it was consequential.

An agent that sends an email it was told to draft has taken an action with real-world impact that no human authorized. Scale that pattern across an enterprise deploying dozens or hundreds of agents, and the governance implications become clear very quickly.

The uncomfortable parallel

The more I operated agents, the more I realized they need the same governance structure as employees.

Not metaphorically. Operationally.

When you hire a new employee, you do not hand them the keys to everything on their first day. You scope their access. You define their responsibilities. You supervise their work. You review their output. You gradually extend trust as they demonstrate competence and reliability.

Agents need exactly the same treatment.

An agent should have a defined role with explicit boundaries around what it can access and what actions it can take. Its work should be reviewed, especially early in deployment. Its access should be scoped to the minimum necessary for its function. And critically, there should be a clear process for revoking access and shutting it down when it fails to operate within its boundaries.

Most organizations are not doing any of this.

Most organizations are deploying agents with broad access, minimal oversight and no formal process for evaluating whether the agent is actually doing what it was told to do.

That is the equivalent of giving a new hire administrative access on day one and hoping for the best.

Human in the loop is not optional

There is a strong pull in the current market toward full automation. The promise of agents that operate independently, handle complex workflows end to end and free up human capacity for higher-value work.

That promise is real, but it is premature for most use cases.

The current generation of models is extraordinarily capable. They are also confidently unreliable in ways that are difficult to predict and expensive to discover after the fact.

Human oversight is not a sign that the technology is immature. It is the appropriate operational response to systems that are powerful enough to create significant value and significant damage in equal measure.

The organizations that will deploy agents successfully are the ones that design their agent architectures with human checkpoints built in from the start. Not as an afterthought. Not as a fallback when something goes wrong. As a core architectural requirement.

Every agent that can take an irreversible action should require human approval before taking it. Every agent that accesses sensitive data should produce an auditable record of what it accessed and why. Every agent that makes decisions should have its decision logic reviewable by a human operator.

This is not a technology problem. It is a governance problem.

The NIST AI Risk Management Framework gets this right. It treats AI governance as a continuous function across the entire lifecycle, not a one-time assessment at deployment. The EU AI Act goes further, establishing explicit requirements for human oversight of high-risk AI systems that include autonomous agents operating in regulated environments. Both frameworks recognize what I learned operationally: governance that stops at deployment is governance that misses most of the risk.

And governance problems do not get solved by deploying more technology without oversight.

What CIOs should do now

The first step is visibility. You cannot govern what you cannot see. Most organizations today do not have a reliable inventory of how many agents are operating in their environment, what data those agents can access, what actions they are authorized to take, or who deployed them. That inventory is the starting point.

The second step is trust boundaries. Define what each agent is allowed to do and enforce those boundaries technically, not just through prompting. Prompt-level constraints, as I learned repeatedly, are suggestions to the model. They are not guarantees. Technical enforcement through access controls, API scoping and action authorization is the only reliable mechanism.

The third step is verification. Do not assume an agent is doing what you told it to do. Verify. Review its outputs. Audit its actions. Test its boundaries. Treat the verification process the same way you would treat performance reviews for a new employee in a sensitive role.

The fourth step is accountability. Every agent should have a human owner. Not the vendor who sold it. Not the platform it runs on. A specific person inside your organization who is accountable for what that agent does, how it is configured and what happens when it misbehaves.

AI agents are going to reshape how enterprises operate. That part is inevitable.

Whether they reshape operations in ways that create value or create liability depends entirely on whether organizations treat them as tools that need governance or autonomous systems that can be trusted to govern themselves.

Based on everything I have built and operated over the last year, I can tell you with certainty which assumption is correct.

Verify. Do not trust until the agent has earned it.

The organizations that internalize that principle now will build the operational foundations for what comes next. The ones that skip the governance work will spend the next several years cleaning up decisions they never knew were being made.

Few CIOs are likely to suddenly decide to give an AI agent broad authority over corporate systems. The more likely path to autonomy is far less dramatic. An agent starts with a narrow task. It proves useful, so another system is connected. It then needs more data, so another permission is added. Then human approval becomes a bottleneck, so the organization removes it, and exceptions that once went to people start being handled automatically.

Each decision may be sensible in isolation. Taken together, however, they can leave an agent with considerably more authority than anyone originally envisioned.

That’s the danger of autonomy creep. And as enterprises move from AI systems that recommend actions to agents that execute them, CIOs need to govern not only what those agents were originally designed to do, but what they have gradually become capable of doing.

It starts with small permissions

Jim Taylor, president and chief product and strategy officer at identity security company RSA, sees a familiar pattern. Organizations typically create an agent for a specific purpose and give it the permissions necessary to accomplish the task. Later, the job expands.

“They just give it tasks, permissions; it just sits there and accumulates things,” he says. “Over time, it develops potentially toxic combinations. It gets too much access and can see too much data. It can do too many things.”

Humans accumulate permissions, too, of course, but organizations have developed processes around employees joining, changing roles, and eventually leaving. Agents may not have the same natural lifecycle. “They never give permissions back,” Taylor says.

Nancy Wang, CTO of 1Password, argues that CIOs should treat each expansion in an agent’s responsibilities as something more consequential than another feature. “Every time you expand the scope of the agent’s effective authority, you’re essentially adding a new authorization boundary,” she says.

That distinction between capability and authority is important. An agent may be technically capable of accessing a system, changing data, or executing a transaction, but that doesn’t mean the enterprise should automatically authorize it to do so.

Success can weaken oversight

Ironically, autonomy creep often occurs when AI works well. Paul Dongha, head of responsible AI and AI strategy at NatWest Group, sees that potential at the bank.

NatWest deployed agents in complaint handling, he says. Parts of the workflow that involve retrieving information from systems can be automated relatively easily, while humans continue to make the final decision on whether to uphold or reject a complaint.

If a system consistently produces the correct result, human reviewers may begin treating approval as a formality. The organization may eventually decide that the check is no longer necessary. “The temptation is to give over to the AI because it’s so reliable,” Dongha says.

NatWest sometimes reduces checking when an AI system demonstrates sufficiently high performance, he adds, but the decision is deliberate rather than incremental. It involves stakeholders and risk managers, and the system continues to be monitored afterward.

That’s a critical distinction for CIOs. Removing an approval step doesn’t necessarily represent poor governance. Removing it because everybody has gradually stopped paying attention to it does.

The warning sign is when a human moves from making a meaningful decision to merely rubber-stamping one — and then entirely disappears from the workflow without anyone reassessing the authority the agent has gained.

Put the controls outside the model

As agents acquire more authority, CIOs can’t rely solely on the agent to respect instructions about what it should and shouldn’t do. At 1Password, Wang advocates what she calls verified loops. Before an agent begins work, humans define its task, which tools it may use, what evidence will demonstrate that it completed the task successfully, and which actions it should never perform.

A separate verifier then compares evidence from the agent’s actions against that human-defined policy. “What you want to avoid here is something recursive where you have a fox guarding a henhouse, or the agent looking over its own work,” Wang says.

The principle applies beyond software development. A model might be instructed never to transfer more than a certain amount of money, delete a production environment, or access particular data. But if violating that rule would have serious consequences, the limit should also exist outside the model.

RSA’s Taylor makes the same distinction from the identity and access side. “Model-level guardrails are necessary, but not sufficient,” he says. “The real limits need to sit outside of that in enterprise control.” He argues for layered controls around agents, including identity policies, provisioning rules, lifecycle policies, governance, monitoring, and attestations.

In other words, CIOs shouldn’t ask an AI agent to exercise restraint where the infrastructure could simply prevent the unauthorized action.

Govern access in real time

Traditional access governance often operates on a schedule. A manager might review an employee’s permissions quarterly and determine whether those permissions are still appropriate. Agents may evolve too quickly for that approach. “Agentic identities are dynamic,” Taylor says. “So we need a mechanism that certifies them continuously, rather than performing periodic reviews.”

If an agent suddenly gains access to another database, for example, that event itself could trigger reassessment. Organizations should determine whether the new permission is appropriate or creates a dangerous combination with access the agent already possesses. “Privilege and authority should never be permanent,” Taylor adds. “They should always be reviewed.”

Wang favors an even more granular approach of task-based authorization. Rather than giving agents standing access or long-lived credentials, organizations can make authorization decisions at runtime and provide short-lived credentials only when required for a particular task. “We espouse no standing privileges,” she says. “No long-lived credentials.”

That effectively applies least privilege at agent speed. A change in task becomes an opportunity to reassess whether the agent still needs the access it’s requesting.

Make someone own the agent

Technical controls, however, don’t eliminate the organizational question of accountability. Taylor believes enterprises should be able to associate agents with a responsible identity. If an employee creates an agent, the organization needs to know who’s responsible for it, what its purpose is, and what happens to it if that person changes roles or leaves. “Every agent action needs to be attributed to a responsible identity,” he says.

Ownership also needs a lifecycle. An agent that remains important after its creator leaves should be transferred to another designated owner rather than simply continuing indefinitely.

Dongha makes another distinction. For AI embedded in a business process, he believes the business owner should remain accountable because that person owns the customer relationship, the process, and often the P&L. “It’s the business owner who has the responsibility because their people run it, and they’re responsible for the customer relationship,” he says.

That creates a useful division of roles for CIOs: IT may provide the infrastructure and controls, but the business can’t outsource accountability for the consequences of autonomous decisions.

Multi-agent systems need system-level controls

The challenge becomes more complicated when agents begin working together. An individual agent might remain entirely within its permitted scope while contributing to an overall outcome nobody intended. One agent generates an output. Another accepts it as input and takes an action. A third agent then acts on the result. Again, the individual actions may all appear legitimate.

Dongha says enterprises therefore need to test not only individual agents but the entire network as a system. “You need something overarching on top of a multi-agent system,” he says.

NatWest is working on an AI control tower that would give authorized users visibility into a network of agents, including messages passed among them and the guardrails surrounding those interactions. Dongha considers governance of multi-agent systems an emerging problem that nobody has fully solved yet.

Wang also points to a related challenge that identity becomes more complicated when agents are ephemeral. An organization may spin up a temporary team of agents, tear it down, and recreate it later. If something goes wrong, the enterprise needs to reconstruct which agent made which decision during which session, and how that action propagated through the system.

That makes traceability increasingly important as autonomous systems become less like individual applications and more like networks of interdependent actors.

Set the bar before the agent reaches it

Before an agent goes into production, Taylor says CIOs should be able to answer several basic questions: Who owns it, what can it do, what’s its maximum authority, how will the organization know if that scope changes, and which actions require a human decision.

Other industries don’t need to replicate the governance machinery of a large bank, but they can borrow one important principle: controls should be built into the system from the beginning. “Governance shouldn’t be a check at the end,” Dongha says. “It needs to be embedded when you build an AI system.”

For CIOs, that may be the central lesson of autonomy creep. The question isn’t simply whether an agent can be trusted today but whether the organization has mechanisms to notice when tomorrow’s agent has quietly acquired authority that nobody ever intended to give it.

The emerging challenge with AI agents isn’t simply preventing attacks from external rogue AI agents; it’s accounting for what your own agents are doing on the inside. AI agents are already rewiring how organizations operate, and we’re only in the first inning. Recent incidents, however, have exposed the risk for individuals and enterprises alike. This is happening because organizations are moving fast and pushing themselves to move even faster. 

In this environment, AI agents have created edge cases that reveal real risks when deployed without sufficient security controls. And further, no one can predict the impact of advances coming over the next six months. All of this reinforces the need to get security fundamentals right. If you’re building with AI agents right now, you know the real challenge isn’t just getting them to run. It’s knowing what happens when they act on their own. That starts with three questions. What agents do I have in my environment? What are they doing? And what are they capable of doing?  

The proliferation problem  

Agentic governance is problematic because enterprises must track a sprawling agent landscape AND ensure each agent is built and deployed securely. Also, agentic adoption is frictionless. Creating an agent is only one API call away from a model provider. Vibe coding means any curious employee can build an agent. This triple proliferation of vendor agents, platform native agents and homegrown agents compounds the problem. 

Visibility is the starting line  

There is the famous security adage that you can’t protect what you can’t see. Right now, most organizations lack basic visibility into what agents are in their environments. According to Anthropic, 82 percent of organizations have discovered unregistered agents in their network, and 57 percent already deploy agents for multi-stage workflows.  

In addition to basic visibility, enterprises also need to understand other related attributes, such as who onboarded the agents, what access privileges they have and how they are behaving as designed. For example, an over-provisioned agent, like an email agent with 50 tools attached, would be suspect and worth investigating.  

Security teams also need one view of their agents across every platform they run on, because each platform has its own permissions and its own telemetry. When each agent has a single record, security teams can connect what it can do on one platform with what it can do on another, and compare both with what the agent was built to do. That’s how seemingly routine activities add up to an attack chain. An agent might read customer records in one cloud, hold a credential for a payments API in another and be able to send messages from a SaaS application. No single platform sees a problem, but together they form a path to move sensitive data out of the company that the agent’s intended scope never required. Similarly, an agent that gains permissions across several platforms over time might never trigger an alert in any one of them, yet the combined record shows how its reach has far exceeded its purpose.   

Governance must extend from build-time to runtime  

Shift-left security helps find model or tool vulnerabilities before deployment, to find risks before an agent goes into production. That’s a necessary first step, but insufficient. Agents create tightly coupled runtime behavior-to-vulnerability. Unlike traditional software, AI agents exhibit behaviors at runtime that can’t necessarily be predicted by inspecting the code in the development phase. Code analysis can’t uncover prompt hijacking, poisoned context or goal drift. 

The model has evolved from “secure the code, then ship it” to continuous behavioral oversight at execution time, layered on top of build-time governance. Pre-deployment security analysis is still required because spec-driven development enforces checkpoints before code is generated. But runtime monitoring is also necessary to detect exposures in the production environment and enforce what agents do versus what they’re scoped to do. 

In practice, this means applying two layers of control to every agent. The first layer is runtime behavioral boundaries. These define what an agent can do. They determine what the agent can reach, what it can return and where it must stop and ask for human approval. This is the enforcement mechanism that operationalizes runtime governance. The second layer is identity controls. These define what an agent is allowed to touch.  

Think of every agent like a scoped, temporary employee with an expiration date. You enforce this by applying familiar IAM principles to agents. Least privilege. Short-lived credentials. Multi-factor authentication for long-lived credentials. Just like an individual contributor engineer shouldn’t get access to the broader team’s payroll, a code generation agent shouldn’t be provisioned to push directly to production environments. An agent designed to answer customer queries shouldn’t get read/write access to the customer database. An IT helpdesk agent doesn’t need domain admin privileges to alter security policies. 

Prioritize by exploitability, not severity  

AI-fueled exploits are breaking down CVE-based severity scoring. The status quo is security teams patching the most critical vulnerabilities first. But attackers don’t always work that way. They can now use agents to chain low-severity vulnerabilities together in ways that render individual scores meaningless while building attack paths that combine for a devastating impact.  

For example, an agent might combine an information disclosure flaw, a misconfigured service account and an unpatched privilege escalation into a single attack path that reaches sensitive data, even though none of those vulnerabilities would individually trigger an urgent patch cycle. If that path leads to customer PII or financial records, organizations face regulatory action under frameworks like GDPR, HIPAA or PCI DSS. If it exposes API keys, service credentials or internal network configurations, the attacker gains a foothold for lateral movement across the environment. 

Context is critical. A vulnerability in network-isolated code is much less dangerous than the same vulnerability in an internet-facing service. Rather than treating every vulnerability in isolation, the better approach is to prioritize based on exploitability within the actual runtime environment. Factor in what’s reachable, what damage an attacker could do and the business impact.  

The new security fundamentals for agents 

The emerging challenge with AI agents is accounting for what your own agents are doing inside your enterprise. In this environment, enterprises need visibility, runtime governance and exploitability-based prioritization. To that end, the security fundamentals you already know still matter. You should already be applying least privilege, short-lived credentials, MFA for the necessary long-lived credentials, automated pipelines and proper testing to secure your corporate environment. These controls limit what an agent can access and what damage it can do if compromised. You will also want to add the agent-specific fundamentals mentioned earlier: visibility, continuous behavioral oversight, identity controls and exploitability-based prioritization. These capabilities tell you what your agents are doing, whether they should be taking those actions and how to intervene when they drift. Together, pre-agentic security fundamentals and agent-specific best practices provide the baseline governance guardrails to secure AI agents. 

Organizations that get this right can accelerate agentic deployments at scale, unlocking innovation while minimizing unintended consequences.  

For the past half century or more, the teams running enterprise software stacks have nurtured a vast ecosystem of tools to keep software humming along. They’ve built CI/CD pipelines, elaborate test suites, code repositories, and tools galore, all with the goal of keeping the systems up and running.

Now AI is throwing a wrench into the whole endeavor. Software is now more complex and sometimes mysteriously unpredictable. Expectations are sky high, but no one is certain what to expect in good conscience.

Luckily, dozens of companies are rushing to build new tools designed to keep agentic stacks on the straight and narrow, delivering value to everyone. The field is rapidly evolving, probably having changed since you started reading this article.

Some of the tools involved in the emerging agent stack aren’t that different from what enterprise teams already have in store. In the end, LLMs are just piles of bits that need to be loaded into a processing chip, just like old-fashioned software. Many of the old tools that watch for overloaded nodes or overpacked disk drives are just as valuable as ever, perhaps more so because LLMs chew through storage even faster. And some of the companies tackling these problems are the same steadfast suppliers who have earned the trust of your DevOps teams; they’ve just extended their own offerings to handle LLMs and AI.

Other challenges, however, are quite different. For example, LLMs are terrifyingly non-deterministic, a real challenge for belts-and-suspenders engineers who traditionally tend the software. Tool vendors are building entirely new systems to handle these somewhat random deviations so that traditional DevOps teams — and their C-suites — don’t have heart attacks.

The field is also rapidly expanding in wildly divergent ways. Some vendors are building tools for developers who are creating new LLMs. Others are concentrating on the challenges that come from worrying about an LLM, day in and day out, in large scale production.

We’ve compiled lists introducing the most intriguing companies developing tools for this rapidly altering space, including tools aimed at AI orchestration; AI observability, aka AgentOps; AI evaluation and benchmarking; and AI trust, guardrail, and governance. We’ve even thrown in some vibe-coding tools for citizen development and AI workflow tools for business processes.

While it was relatively easy to split the field into distinct lists, that process will only grow more challenging because the tools — and categories — are starting to overlap. As vendors evolve their strategies, this overlap will grow further, with a variety of entries tackling a wider range of challenges, vying for a greater percentage of CIOs’ AI middleware and operational budgets.

To help you make sense of this complex tool landscape, here are the most common categories vendors are using to identify what their products do, each of which maps to what are becoming the key components IT needs to keep their AI and agentic stacks productive and humming.

Observability

Vendors often use the term “observability” to refer to the most basic of data collection work — that is, testing to make sure the various machines are running, the RAM isn’t too full, latency is acceptable, and the logs aren’t filled with error codes.

Watching these metrics isn’t much different from caring for an old-school collection of microservices, and often the tools aren’t much different, either. Many of the most popular tools are the same dominant market leaders that have been extended to collect data from the various GPUs, too.

Leading vendors: DataDog, Dynatrace, New Relic, Splunk
Upstarts: Ariz, LangSmith, Braintrust, Galileo
Open-source alternatives: OpenTelemetry, Phoenix, Langfuse

Cost engineering

Many AI pipelines are often more complex than regular microservice architectures, and that means they need someone to carefully watch the bill. Plus, as attention to AI ROI rises, capturing spend is essential — especially given how quickly AI tokens can add up.

Cost engineering tools can integrate information from various portals like OpenRouter and compare it against internal hardware. Sometimes breaking down these comparisons is difficult because the right way to allocate costs to each user isn’t as obvious, even though they’re often just one number. These accounting operations tools will become even more important as LLM bills grow.

Leading vendors: CloudZero, Finout
Upstarts:  Amnic, Bifrost, Portkey
Open-source alternatives: OpenCost, Langfuse

AgentOps

When metrics and telemetry begin to focus on the LLM, the word “AgentOps” starts to appear. It’s just like DevOps but it’s concerned with watching token counts, tracking the performance of RAG databases, and collecting traces for debugging purposes.

When the work is truly agentic, AgentOps must also juggle the state that agents generate while looping through their various activities. Agents aren’t just a one-and-done call to an LLM. They’re software packages that need to store information as they work toward a final answer.

This job of an AgentsOps team gets trickier because these areas are where the models often behave non-deterministically. Is some model hallucinating? Are an agent’s prompt chains running to completion? The tools in this space track all these metrics, often in a dashboard with nice graphs that make problems stand out.

Leading vendors: Databricks, Datadog, Dynatrace
Upstarts: Arize, Braintrust, Comet
Open-source alternatives: Langfus, LlamaIndex

Evaluation

The buzzword “evaluation” starts to appear alongside tools that are used in the earlier stages of development, when various models and prompt combinations need to be tested again and again.

In many cases, these tools use another model to evaluate the quality of the answer from the first model, as well as to keep an eye out for potential hallucinations or other failures. In some cases, the tools query humans who sit in the evaluation loop to give their opinions of how well the model is performing. Good tools can watch the agents continuously and reroute any sketchy behavior to the human-in-the-loop.

Many of the vendors use the word “evaluation” to describe measuring just how well a model is performing for a particular business application. That is, how well it’s working for what your boss wants it to do.

Leading vendors: Databricks, SuperAnnotate, Scale AI
Upstarts: Braintrust, Galileo AI, MaximAI
Open-source alternatives: Langfuse, Prompfoo, Tagas

Benchmarking

The word “benchmarking” is similar to “evaluation” in AI vendor literature, but it is usually found more in the tools used by the creators of the models. There are a number of public benchmarks filled with standard queries, and the model developers rank themselves on how well they do solving these problems. This often generates a raw score that is then compared to another’s raw number. Some teams create their own benchmarking problem sets and use them during evaluation to assess how well models might meet their needs.

Leading vendors: Hugging Face, Databricks
Upstarts: Reka AI, Patronus AI
Open-source alternatives: Promptfoo, DeepEval

Trust

The tools that wear the “trust” benchmark are often meant to help evaluate how well a model is protecting the needs of any business that is deploying it. That is, can they trust it to deliver accurate and useful answers without hallucinations, data leakage, inappropriate behavior, or worse? Prompts with malicious instructions are especially tricky to detect. Some enterprises want a trust score that measures the consistency of the answers. Overall, tools that target the “trust” metric are often utilized more during development and early deployment.

Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard

Guardrails

The word “guardrails” is usually applied to tools that are part of the deployment stack. That is, they sit as watchdogs and make sure the LLM doesn’t spin out of control. They also ensure the model is trustworthy, but they’re more tuned to operating in real-time than those aimed at trust, as outlined above.

These tools are often filled with fixed rules, such as scanning for social security numbers or watching for noxious words. The challenges get trickier because agents are executing software, and that means the guardrails must do more than analyze outputs. They also need to manage the execution environment, often running in a sandbox, and watch for exploits and other malicious behavior. They need to validate and double-check not just code, but the OS-level commands for manipulating files and processes.

Leading vendors: Azure AI Content Safety, Bedrock Guardrails, Vertex AI Search
Upstarts: Patronus AI, Arthur AI
Open-source alternatives: NeMo Guardrails, Llama Guard

Context

Many systems rely on a vector database to store embedded data to chunks of trustworthy documents. These RAG databases need much of the same care as regular databases, and the tools for their care are often similar. PostgreSQL, for example, has a vector addition that needs the same kind of scrutiny from a database administrator as a regular PostgreSQL database.

Leading vendors: MongoDB, Elasticsearch
Upstarts: Pinecone, Weaviate
Open-source alternatives: Milvus, PostgreSQL

Gateway routing

Increasingly, AI systems employ models in constellations, and a gateway router is what examines a prompt to determine which model or agent is best for answering the query. Moreover, as agents undertake their work, the router can distribute the agent’s various subcalls to the fastest, cheapest, or most accurate LLM or subagent.

Watching over this stage in the pipeline requires a different set of intelligence because the questions can often involve both applicability and cost. Sometimes this layer must balance the extra cost versus the need for a better answer.

Leading vendors: Cloudflare, AWS, Google Cloud, Azure
Upstarts: OpenRouter, ModelsLab, Respan
Open-source alternatives: LangChain, Bifrost, LiteLLM

Access control

Who gets to use the model? Who is paying? Many access control tools are the same ones that were part of standard APIs or web services. Is there an API key? Is the account below its budget? The tools watch for overspending because both the customer and the boss want to make sure the money keeps flowing and no tokens are wasted.

Leading vendors: Auth0, Apigee
Upstarts: Amnic, Portfoundry
Open-source alternatives: LiteLLM, Bifrost

Compliance

Some industries have explicit compliance requirements dictated by law. Some have their own internal controls. Tools devoted to compliance are often a subset of the ones devoted to guardrails. But they’re often treated separately from guardrail tools because they’re more likely to be of interest to lawyers and that makes the stakes higher.

Leading vendors: ServiceNow, IBM
Upstarts: VerifyWise, Norm AI
Open-source alternatives: Aether, Giskard

Fresh off its $1.1 billion acquisition of AI startup Sana, Workday is making another bet on enterprise AI, this time targeting the governance challenge. The company said it will acquire agentic identity platform C1.ai to help customers manage and secure AI agents operating across enterprise applications.

The acquisition will build on Workday’s Agent System of Record, which gives enterprises a centralized view of their AI agents, by adding controls over what those agents can access across Workday and other enterprise systems, the company said.

That would allow enterprises to grant agents the permissions they need to perform specific tasks and revoke them when those tasks are completed, rather than risk agents retaining access to data and applications they no longer need, it added.

The combined capabilities could also help streamline access management for employees as they join an organization, change roles, or leave, reducing the manual work involved in managing permissions while limiting potential security gaps, it further said.

These governance and identity management capabilities become increasingly important as Workday, alongside the acquisition, also continues to expand the role of AI agents across its suite.

During the week, the company announced several updates to Sana, including making it the new home interface for its applications.

The interface brings tasks, decisions, meetings, and agent activity into one place, allowing employees to use natural-language prompts to get Workday agents to complete tasks rather than simply retrieve information, the company said.

The company also extended Sana into third-party tools, including Microsoft Teams, Slack, and Microsoft Copilot, allowing employees to invoke Workday capabilities from the applications they already use.

That expanded ability to invoke Workday agents across applications underscores the need for enterprises to govern what these agents can access and do across systems. C1.ai’s acquisition is expected to close by the end of January.

IT and security leaders are well-versed in the risks of unauthorized integrations, misconfigured tools and vendor compliance. Now, AI has introduced a separate and harder-to-detect problem.

Across every function, employees use provisioned AI tools to generate production-grade code, build automated workflows and deploy agents to support business processes. That experimentation is happening without visibility into what others have built, or the structural risks accumulating as a result.

The governance models enterprise organizations relied on even six months ago were not designed for this moment, or what lies ahead. We’ve focused so intently on which AI risks to govern that we never stopped to reimagine what operating model should underpin the governing.

How invisible risk accumulates

A predictable operating bottleneck plays out inside large enterprises. A team needs AI support, but the formal channel is slow, backlogged or unclear. So, the team builds something independently. That instinct is right, but the structure is not.

The costs from this kind of inefficiency can compound quickly. Effort gets duplicated across departments with no visibility into each other’s work. Siloed tooling emerges with no shared standards, and security and data handling practices diverge. People get pulled away from their core responsibilities to maintain AI tools that three other teams independently built last quarter.

Meanwhile, the governance function stays busy reviewing the growing queue it can see: integrations passing through approvals and vendors that appeared in a procurement request.

A misconfigured tool that made it through review is a governed risk; a production workflow a finance team built over three weekends with no oversight is still invisible. At most organizations, this second category of unseen work is growing, and the scale of what’s coming makes the gap harder to ignore. By 2028, Gartner predicts that the average global Fortune 500 enterprise will use more than 150,000 agents.

Build an operating model that scales AI governance

Experimentation is essential for finding valuable AI use cases, which means suppressing experimentation through tight controls is the wrong approach. Tightening controls also often pushes experimentation underground.

To scale governance effectively without creating new problems, you need to accommodate AI experimentation. Here are three ways to get there:

1. Establish a central team focused on policy and standards. Designate a centralized AI team as the single interface to support risks surfaced by your security, legal and compliance functions. This team sets technical guardrails, owns vendor selection and creates reusable patterns that every function across the organization can draw from. Its job is to make responsible AI work the default so that individual review becomes the exception rather than the rule.
2. Embed AI transformation owners inside each function. These embedded owners that work day-to-day inside a function but report to a centralized governance team are close enough to understand context and also apply controls consistently. They serve as the single intake point for AI requests within their function, so instead of teams building in isolation, there is always someone who can triage, prioritize and connect the work back to what already exists elsewhere. The reason this works where formal channels fail is speed: when the sanctioned path is faster than building alone, teams stop routing around it.
3. Measure governance by what compounds. A good signal that your governance is enabling sanctioned AI experimentation is how much work across your organization is compounding in value. In other words, look at the reuse of company-sanctioned AI use cases: a pattern built for finance getting picked up by operations, a workflow proven in one team becoming the default in three others. A dropping volume of IT requests combined with compounding (and visible) AI usage tells you experimentation is both maturing and staying visible.

The costs of waiting will compound

As AI compute costs swallow IT budgets, the cost of a poor operating model does not become visible until it is expensive to fix. Shadow tooling that starts as a workaround becomes load-bearing infrastructure. Teams that build in isolation develop technical debt through approaches that are hard to standardize later. There’s a lot more to undo.

The organizations treating AI governance as a gatekeeping function have missed the point. AI needs governance, but how we see the role of governance needs to change. The biggest risks live in the work that never enters the queue, and the only way to govern that work is to be present where it happens, before it compounds into something much harder to fix. AI moves fast; we need governance standards that travel with it, rather than chasing behind.

In an August 2026 Gartner report, 40 percent of C-level executives say their organizations have “operationalized AI in some business processes,” yet only 13 percent say they have “achieved measurable results and are scaling AI across the organization.”

This disconnect reveals a frequent reality: AI may have been “hired”, but companies haven’t onboarded it effectively.

Research from HR analyst firm The Brandon Hall Group found that organizations with a strong people onboarding process improve new hire productivity by over 70 percent. Similarly, the first period of AI’s “employment” in a specific role largely determines how effectively it will perform and its perceived “growth potential”. But many tech executives are ignoring this crucial stage of the implementation process and are left with AI systems that cannot accomplish the complex tasks they’re assigned.

The most successful companies are those that treat AI like their newest hire instead of a long-tenured team member.

Onboarding AI means establishing the right context and connections

Establishing the AI’s persona, the context in which it operates, how it assesses itself and the people and systems to which it is connected are what define implementation success. Usefulness quickly diminishes when even one of these elements is left out of the onboarding process.

For example, we recently created an internal agent called ‘WorkRadar’ that surfaces missed tasks and answers questions like: “I’ve been on PTO, what’s the summary of what happened, what should I skim and what should I review in detail?” We gave this agent all potentially useful context by connecting it to Outlook, Microsoft Teams, Confluence, Asana, SharePoint, meeting notes and our human resource information system (HRIS) so it could determine the responsibilities of the person invoking it and infer topics of interest.

Much like a new employee, however, WorkRadar couldn’t derive the meaning behind this information without further guidance. When I first used the agent, it did not distinguish between strategic priorities and tactical work and also missed initiatives I sponsor across departments, outside the typical hierarchy. This version of WorkRadar did not know me enough to be useful and I was better off doing the work myself.

Instead of leaving WorkRadar to flail as-is, we further onboarded it by enabling users to provide the agent with a personalized description more detailed than it could infer through company documents alone. This included their own responsibilities, operating level, current priorities, key stakeholders for specific initiatives and communication style. For most workers, this is less than one page of short bullets and has made a significant difference. Now, many of our employees rely on WorkRadar like they would a stellar executive assistant, to ensure relevant outstanding to-do items don’t fall through the cracks and smooth their return from PTO.

Five principles that define AI success

WorkRadar’s improved onboarding illustrates the essential elements for priming AI. Managers should define each AI agent’s:

1. Role

Articulate the AI agent’s position and purpose in the organization; everything else derives from there. Tech implementers should determine if the AI agent should act as an employee avatar with the same access rights, a simpler approach, or act under its own identity, which is increasingly becoming best practice for governance, risk and compliance reasons.

Implementers should also decide whether the AI agent “sits” within their infrastructure or their vendor’s. Tech executives can determine this by assessing their companies’ business context, security posture and preference for hands-on involvement with agents. For companies like ours with more than one agentic stack, we have agents that sit in both places.

Finally, they should define which employee will be an agent’s supervisor, and the specific contexts in which AI stops for human review. It’s a huge mistake to have no ownership of a tech component and no human in the action or feedback loop to improve performance and address errors.

2. Remit

Articulate the outcomes this agent should achieve, then identify the systems it’ll need to access for information and to carry out its work. First, tech implementers must give AI read-write access to the systems it will modify via MCPs, tools, skills, APIs, just to name a few methods. This boundary-setting will determine AI’s “sphere of influence” (what we want it to access) and its “blast radius” (the systems we accept it may access and must protect from misuse).

To give AI guidance, employers must also give AI read access to job-relevant corporate directories and policies, so the agent knows how to navigate the organization and where to direct communication, as well as company strategy documents and project management trackers, so it can understand the “why” behind its work.

Control and security are a complex topic but, at a high level, implementers must also provide access to lists of governance and compliance limitations to define the boundaries by which agents must abide, plus tradecraft standards and procedures so the agent acts in accordance with industry and corporate best practices.

3. Personality

A huge amount of information could, and eventually must, go into an agent’s context. However, Agents have finite memory and focus, so, like people, they are easily overwhelmed with too much information. So, tech implementers must pick and choose which parts of the agent’s context are permanent and imbued in every task versus derived just-in-time by the agent.

This is the art and science of context engineering and it’s evolving rapidly. Elements that are omnipresent in all agent actions – persona, strategies, guardrails, etc. – constitute its system prompt. They’re written directly into the agent’s context by the user, system administrator and the agentic harness on a per-job role­ basis.

Meanwhile, employers should also define elements the agent derives in real-time. Frequently, the performance of an agent is determined in this probabilistic work, which is why delineating what’s permanent versus conjured in the moment can make or break an agent’s efficacy.

It’s a balancing act that’s costly on both ends: too much context, and the agent chases expensive side quests. Too little, and the agent burns resources discovering what it needs. Best practices ebb and flow, with foundation companies like Anthropic recently advocating for brevity, trusting increasingly stronger models to figure it out.

4. Performance review

Give AI the necessary tools and context to assess itself. Otherwise, the agent will be blind to its actual impact and require too much course-correction.

Ideally, employers give the agent a tool to capture and update new memories, including condensed feedback it has received or epiphanies it discovered itself. Ensuring an agent has this learnings memory bank will make it more performant over time since it does not have to repeatedly solve the same problems and can apply context across disparate tasks.

Finally, on a regular basis, the agent’s supervisor must evaluate its work by looking at actual outputs and giving the agent examples of its best and worst work so it can understand actions to continue and stop, respectively. These good and bad examples can be generalized and bundled, and supervisors can then test the AI against them to continually adjust for behavioral shifts. This may be the most neglected activity, yet the one most likely to turn a mediocre agent into a valuable colleague.

5. Teamwork

Lastly, permit AI to access specific communication surfaces like internal email and messenger platforms and interface with employees. For multi-agent orchestration, employers will have to open communication among agent “colleagues.” Each agent needs an awareness of collaborators and means to exchange messages and share files, all while continuing to involve human supervisors, who won’t inherit any errors that upstream agents might pass on to others. Grok and Hermes debuted bots with this capability in August 2026.

Pearl

Even with context, AI is still missing key working principles

Two additional considerations make AI that works like a veteran instead of a new employee.

First, tech implementers should realize AI’s memory is less focused than humans’. AI quickly becomes confused about core information versus extra details and can even lose parts of its “knowledge” because its effective context window is still only about 200K tokens. AI is also prone to “anchoring,” persistently latching onto a throwaway comment or early thought.

The implication for agentic AI users is to start fresh sessions much more frequently than might feel natural or to tap into continually evolving techniques to manage context, including compaction, truncation and summarization. Agentic platform makers have started to build this into their harnesses, but it’s still far from “set it and forget it”.

Second, and perhaps most frustrating of all, is AI’s myopic focus on its task without care or curiosity about the greater context surrounding it. This behavior causes it to implement solutions that miss important success criteria or violate critical constraints. In this sense, AI grants one’s wishes a bit like a genie, so be careful what you ask for and how you ask for it!

For example, left unprompted, coding agents are a bit too quick to jump to implementation. One architect solved this by creating a workflow that forces the agent to interview the developer first, sometimes asking upwards of 40 clarifying questions, to surface missing requirements and edge cases before writing a single line. Some agentic coding harnesses now perform developer interviews, but there’s still room for improvement.

These deeper techniques highlight that agents need more than a good initial onboarding; they need an iterative management approach, just like humans.

Many CIOs today are looking to use AI more for innovation and modernization than sheer productivity.

Assume AI already knows the job and you will be frustrated when it falls short. Treat it like a new hire and it will grow into one of your best.

Eager to incorporate agentic AI into their work, employees are taking matters into their own hands, creating their own AI agents, regardless of organizational approval, and a not insignificant number of those creations could have devastating consequences if IT leaders don’t get ahead of the trend.

According to a recent survey by engineering staffing firm Howdy.com, two-thirds of knowledge workers that use AI also use agents, and 16% have begun to build their own.

About a quarter of agent users have had to deactivate them, however, largely because the agent’s outputs were inconsistent or the agent never worked. But 7% of knowledge workers have had to kill agents that went rogue.

The findings put a point on the potential repercussions of shadow agent creation and even sanctioned citizen AI strategies that encourage business users to roll their own agents.

While the survey didn’t ask whether the employees building their own agents were doing it on their own, Frank Licea, founder of Howdy.com, sees that many aren’t waiting for permission.

“In our internal teams, we’ve had to, not put the brakes on, but ask people to hold on,” he says. “Before they start jumping in and building their own agents, we’ve had to put guardrails and system guides in place so that we can do this safely.”

For what Licea has seen, employees are excited to automate boring manual tasks by building their own agents. “It mostly starts unsanctioned because there are tech-forward folks in any organization who are just naturally curious,” he says. “Or maybe even intelligently lazy. They want to automate their work.”

In recent months, Howdy.com employees began telling each other how they automated manual tasks, without waiting for the company’s IT team to approve the uses, Licea says.

“The way it happened for us is that folks would come up to us like just beaming and excited about something that they automated or fixed a long-standing problem that the engineering team never had on their roadmap,” he says. “Once that started happening, then somebody from the tech side gets in and they start asking questions like, ‘What are the API keys here? Are you accessing encrypted data? What are the SOC2 implications?’”

Cautious embrace

Luis Mesas, VP of product engineering at software engineering provider Sngular, believes it’s inevitable that employees will experiment with building AI agents to automate their work.

“Employees are going to create their own agents whether companies formally sanction it or not, because the barrier to building these tools is getting lower, and engineering teams are already being asked to do more with less,” he says. “The bigger concern for me is not that an employee creates an agent, but what happens when that agent starts connecting to company systems and data without the same controls that would normally exist around a piece of software.”

IT leaders need to understand how employee-created agents are connecting to internal systems and data, he says. If an employee creates an agent that can pull customer information from a CRM, update records, or move information between systems, IT leaders should know it’s happening, he adds.

“I want to know exactly what that agent can access and whether the employee should have that level of access in the first place,” Mesas says. “That is where something that started as an individual productivity tool becomes an enterprise technology issue.”

Andy Sen, CTO at B2B commerce platform provider AppDirect, believes employees should be encouraged to build agents — but within guardrails and established policies.

“Employees are almost certainly creating their own agents, and it would be naive to think otherwise,” he says. “Shadow IT has always existed, and shadow agents are just the latest version of it.”

The answer isn’t to lock things down, however, he says. At AppDirect, Sen gives employees a “proper digital playground,” and in response to employee demand, his team built an AI-powered coding service called Devs.AI.

“We built Devs.AI so our employees can build and innovate freely, but IT has full visibility into usage and costs and the assurance that company data isn’t being leaked to the world,” he adds.

Citizen AI strategies, like the one at AppDirect, are on the rise, but require consistent monitoring. If one in four employee agents are deactivated, for example, IT leaders should pay attention, Sen says.

“Failing and learning is fine,” he adds. “Failing silently is not. You need visibility into what your agents are doing before a small problem becomes a serious one.”

Still, Sen doubted reports of a small percentage of employee-created agents going rogue.

“When an agent goes rogue, that usually means the agent did exactly what it was told, and what it was told wasn’t right,” he says. “That’s on the creator, not the agent.”

Creating a path for good ideas

Andrew Sales, chief methodologist at agile methodology vendor Scaled Agile, also sees widespread agent deactivations as a strong signal for IT leaders to investigate.

“When agents are built quickly and outside a formal model, there often aren’t clear boundaries around what they can do, what data they can reach, or when a human needs to step in,” he says. “So it’s not surprising that some have had to be shut down.”

AI governance is important, but it shouldn’t focus only on preventing bad outcomes, Sales notes.

“It should also create a path for good ideas to move from individual experimentation into enterprise capability,” he explains. “Organizations need a way to identify what’s working, standardize it, make it safe, and scale it responsibly across teams. Done well, that same framework that catches the agents that go wrong is what lets the ones that go right spread.”

Most enterprises govern AI the way they would any other IT rollout: deploy dashboards, institute policies, and conduct periodic reviews. Having the right AI governance tools in place and addressing gaps is important, but IT executives should be more concerned about rethinking workflows and establishing ownership.

“Accountability usually breaks down first,” says Beena Ammanath, executive director of the Global Deloitte AI Institute. “Teams deploy AI faster than their organizations can define who owns risk, approvals, monitoring, and outcomes,” she adds, which results in inconsistent use, fragmented controls, and limitations in scale.

Enterprises need to redesign jobs and workflows, not just the tooling, yet 84% of organizations the Institute surveyed haven’t adjusted jobs for AI, and just 21% have a mature AI governance framework in place despite rapid adoption, according to Deloitte’s 2026 State of AI in the Enterprise report.

“Technology can provide visibility and controls, but it can’t replace leadership,” says Rajesh Arora, chief data and analytics officer at Principal Financial Group. “The organizations that scale AI most effectively are the ones that treat accountability as a business responsibility, not a technology responsibility.”

One year ago Principal was focused on building its ethical and responsible AI framework, its inventory, and governance processes to support responsible adoption. Today the firm has assessed nearly 200 use cases. “Governance is increasingly embedded in those investment and operating decisions, rather than applied as a separate review at the end,” he says.

Costly, compounding mistakes

Some organizations have already run into issues with respect to inadequate governance. A 2024 Stanford RegLab study found that AI-assisted research tools offered by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) hallucinated between 17% and 33% of the time when working on legal queries, while the Damien Charlotin AI Hallucination Cases Database has identified some 2,041cases of fabrications so far. In at least one legal case (Couvrette v. Wisnovsky) this resulted in nearly $95,000 in sanctions.

Cases like this one are likely the result of an accountability failure. It may be that no one actually owned the job of verifying the AI’s output before it went to the court. But if a single instance of a static AI output could get past a review today, how will organizations deal with autonomous, multi-step agentic AI systems going forward?

Most use cases at Principal are still human-led, with AI providing decision support only, says Arora. “But we’re preparing for a future where agentic and more autonomous capabilities become increasingly common, and those systems raise new questions around accountability, monitoring, and control,” he says. “The goal isn’t to predict every future use case, but to build adaptable governance processes that can support innovation while maintaining trust and transparency.”

Agentic automation raises the stakes

When things go south with agentic AI today, the consequences usually aren’t devastating because in most cases, the outputs are designed to provide insights, rather than take automated actions. That’s already starting to change, however.

Today, many AI failures are like the one described above where someone failed to catch a single bad output. But agentic AI will gradually take more control of workflow tasks in the enterprise, according to Forrester Research. As those agents act autonomously, crossing multiple process steps along the way, small errors will compound before any periodic review can catch them, so governance must be built into every step in the process.

According to Microsoft Research’s VeriTrail project, catching hallucinations in a single final output isn’t sufficient in multi-step AI workflows. The provenance must be traced back through the intermediate outputs, because an agent chains one call into the next, and an error early in that chain can propagate through several steps before a human sees the finished result. Traditional review, built to check outputs rather than the chains that produced them, isn’t designed to catch that.

So business teams can’t be accountable for agent deployments if they can’t verify performance at runtime. “If the agent is goal-oriented, there could be errors in the workflow process that indicate unwanted behavior or skipped steps that the business wouldn’t be able to detect if they only look at the outcome,” says Lauren Kornutick, senior director analyst, AI governance at Gartner.

Power management company Eaton, for instance, has already deployed multi-step agentic workflows without a human in the loop. “We tend to be risk-conscious, not risk-averse, about autonomous workflows,” says Ross Schalmo, the company’s VP and chief data and AI officer. For example, one autonomous agent responds to supplier payment status inquiries by authenticating the requestor, matching purchase orders and invoices, and returning payment information.

Schalmo’s team engineers such applications to be as deterministic as possible, while the non-deterministic, LLM parts of the program are isolated and continuously evaluated against expected outputs to catch model drift. “When there’s a human in the loop, we can be a lot more permissive,” he says. “Anything autonomous, without a follow-on human step takes longer to develop because we have to think about where to use the generative portion of AI versus where we can still use deterministic tooling.”

Breakthru Beverage Group is still early in its AI journey, but EVP and CIO Glenn Remoreras is well aware of the issues around autonomous agentic AI. “It’s people, process, and culture first — to me that’s the bigger play,” he says. The beverage wholesaler has updated use policies, instituted mandatory AI e-learning programs, and is blocking use of unsanctioned AI tools.

But to successfully manage active, agentic AI, where the AI takes actions autonomously across multi-step workflows, users will need more than basic training in AI, says Craig Le Clair, VP and principal analyst at Forrester Research. “AI will take on more of the work and move the human out of duct tape roles that patch around vague systems, poorly designed interfaces, and poorly integrated data,” he says. “The end game is the agent managing and orchestrating the process — that’s the redesigned operating model.”

Humans, of course, will still need to play a role, although today many employees don’t yet have the required skill sets, and currently that point isn’t getting a lot of attention.

AI supervision skills will be needed for agentic oversight, threshold management, confidence intervals, and root cause analysis, for instance. “Figuring out why an agent hallucinated or got stuck in an execution loop — whether due to poor input data or contradictory rules — and how to refine instructions, that’s the direction,” Le Clair says.

Who’s governing anyway?

AI governance is also complicated by the fact that AI projects exist both inside and outside the IT organization. That leaves governance work distributed across CIOs, data and analytics leads, AI leads, IT security, legal, and other players, with no consistent owner, says Kornutick. “It becomes a multi-role orchestration problem of how to govern this, who owns this, and who’s responsible for which component,” she says.

In response, some organizations have brought in a chief AI officer. Twenty-one percent of organizations now have already done so, says Le Clair, adding that roughly 24% of enterprises will move AI and automation governance outside the CIO’s office over the next year to deal with the risk of fragmentation.

Arora, who reports to CIO Kathy Kay, is the focal point at Principal. “My team establishes the standards, oversight, and governance process, but AI is too embedded in the business to be owned by a single function,” he adds. They collaborate with leaders from the business as well as the legal, compliance, risk, and technology organizations.

The challenge, therefore, lies in deciding who’s responsible for controls, model sprawl, and token economics. “We’re seeing a cooperative, co-model where IT focuses on the hard rails — cybersecurity, performance management, infrastructure reliability, and availability,” Le Clair says, while the CAIO focuses on soft rails, such as context engineering, the effects on employees as agents move to the center of the process and start taking over the middleware function humans used to perform, and connecting disparate systems and poorly designed interfaces.

At Eaton, governance is built on a three-legged stool between Schalmo’s organization, cybersecurity, and legal, while enterprise architecture weighs in on tech foundation decisions. The company also assigns ownership of an AI project only after a successful pilot, while IT performs a custodian or stewardship role. “Once that agent is live and operational, it shows up in an org chart, reporting to someone in the organization,” he says.

At Breakthru Beverage, Remoreras formed and leads an AI governance team that reports to the board and co-leads with the VP of data and AI a governance council that reviews and approves AI projects from SVP- and VP-level product owners. “I’m seen as the owner of our AI portfolio, but it’s co-leadership with my peers on the executive team,” he says.

Closing the tools gap

With regard to AI management platforms, says Le Clair, none of the platforms building AI agents have an adequate control plane today to give the visibility needed to move to truly agentic solutions. The most advanced planes are walled gardens. For example, Microsoft’s agent registry will show every agent built across its own platforms, says Le Clair, but not agents built on Salesforce or AWS.

“You’ve got maybe 50 vendors trying to be pure-play control-plane providers, plus the hyperscalers, portfolio companies, and startups building agentic frameworks,” he adds. “There’s no consensus yet on what’s even most important to control, and there’s confusion about requirements.”

Most AI governance tools available today are limited, Schalmo says. “We’ve looked at four or five different external vendors for AI governance and AI control-tower-type solutions, and a lot of them fall down on discovery of AI built by someone else’s solutions,” he says. “That’s a huge gap for me. Either I have to buy governance solutions from almost every piece of software we interact with, which gets really expensive, or I have major gaps in my portfolio I just don’t have visibility into.”

Currently, he’s looking at using a cyber vendor’s AI component for discoverability, going with at least one major firm that provides control-tower capability, and using AI to build another governance layer on top of that.

Organizations clearly need agent registries and agent lifecycle management but it’s less clear what they require beyond that. “How deep does explainability need to go when decisions are made at runtime that weren’t articulated at design time?” Le Clair asks. “There’s real vagueness in the requirements. Vendors have very well-tuned pitches that promise to do anything you want, but buyers often don’t know what they need in the first place, and there are real tool gaps. This is the major problem the industry is actively trying to solve. It’ll just take time.”

But those aren’t Arora’s biggest concerns. “Most organizations can acquire tools,” he says. “Far fewer have clear ownership, decision rights, and accountability embedded across the business.”

Speed versus governance

There’s a perception that governance slows down AI projects because it can’t keep up with the rapid pace of adoption and advancement. A 2026 GitLab AI accountability survey of developers and technology buyers found that 92% of organizations reported some form of governance challenge with respect to governing AI-generated code, and 34% of organizations that experienced an AI-related incident couldn’t determine after the fact whether AI-generated code was the cause.

Similarly, Faros AI’s telemetry analysis of 22,000 developers in its 2026 Engineering Report discovered that developers with high AI adoption rates completed 34% more tasks and 66% more epics, but the median pull-request review time rose by 441%, a nearly five-fold increase.

Is it the case that governance and speed are inherently at odds, and that human review simply can’t keep up? That’s a false choice, says Ammanath. Rather, it shows what happens when both aren’t designed together. “When done well, governance accelerates scaling by giving teams clear guardrails from the start,” she says. “What slows organizations down is retrofitting governance after broad deployment.”

Arora adds that controls don’t slow down innovation but protect the conditions that allow innovation to scale responsibly. “Our approach has always been to move with urgency, but not at the expense of trust,” he says.

How to proceed

For Principal, investing in workforce readiness was one of its biggest investments, helping employees understand both the opportunities and responsibilities that come with these tools, Arora says.

But organizations shouldn’t stop there. Redesigning workflows and decision-making structures is essential before scaling, says Ammanath. “Many organizations have policies and technology in place but lack clear decision rights or governance embedded into daily work,” she says.

Operating model changes take more time than training, and require executive sponsorship and cross-functional coordination. “Without workflow redesign, a workforce may understand AI but lack clarity on when to use it, how to govern it, and how daily responsibilities change,” she adds. “The organizations seeing the strongest results pair broad AI education with role-specific training, workflow redesign, and clear accountability for outcomes.”

Make sure you’ve thoroughly considered the organizational change management aspect of agentic AI projects, says Schalmo at Eaton. His team developed an automated workflow for processing complex bid requests to save the sales team time, but piloted it in a country with lower labor costs than the US. That challenged the economics of the project, and those individuals only dealt with higher-volume, lower-complexity cases.

The lesson: “Lead from a people-first standpoint, and understand the workflow and what’s impacted — whether it’s relieving drudgery and freeing someone for higher-order thinking or customer time, or impacting an actual support function’s role,” he says. “And make sure there’s a viable path to business value.”

An adaptive governance process is key to keeping up with the technology, Kornutick at Gartner adds. She recommends categorizing AI projects following a stoplight approach. Green light projects, such as for personal assistants, present acceptable risks. “We know there’s risk associated with it, but we’re willing to take it,” she says. Yellow-light projects need guardrails in place around such things as PII, or for those triggering a privacy-by-design process, or a security review of a new application. Red light projects need an architectural review before proceeding.

“Think about those use cases as business-transformation processes where you’re fully redesigning workflows, or considering replacing tasks individuals do and shifting them to supervising the AI doing those tasks instead,” she says.

Schalmo’s team does risk tiering based on the EU AI Act’s four-tier risk framework to determine the appropriate level of governance scrutiny for each project. Low-risk requests are automatically provisioned after the requestor fills out a questionnaire, while high-risk cases get a full architectural review.

“Treat governance as the foundation for scaling, not a constraint on speed,” says Ammanath. “Before expanding agentic AI, organizations should define what decisions agents can make, where human oversight is required, who’s accountable, and how performance and risk will be monitored.”

The hardest part, says Arora, is helping an entire organization evolve at the pace of technology. “Successful AI adoption is ultimately a business transformation challenge,” he says. “It requires leadership alignment, clear accountability, workforce readiness, strong governance, and ongoing change management. Long-term trust will belong to organizations that can translate technology into sustainable business value.”

The latest warnings about artificial intelligence are difficult to brush aside. Anthropic researcher Jacob Coxon resigned last month, walking away shortly before his equity was due to vest, and publicly argued that leading AI companies are advancing toward increasingly capable systems without adequate assurance that those systems will remain controllable. His resignation landed amid a broader argument over the probability that advanced AI could eventually cause catastrophic or even existential harm. Axios on Coxon’s resignation and warning

It is easy to see why the conversation gets pulled toward extinction. “Could AI kill humanity?” is a much more arresting question than “Are our escalation procedures fast enough?” But for CIOs, CISOs and other leaders responsible for systems that actually have to work, the second question may be more useful. AI does not have to become superintelligent, conscious, hostile or impossible to shut down for governance to fail. Governance can fail much earlier, when a system is given enough capability, access, autonomy or delegated authority that the organization can no longer intervene reliably before consequences arrive. That is not primarily a science-fiction problem. It is an operating-model problem with an engineering component.

The evidence is concerning without being apocalyptic

The extinction discussion deserves some discipline. The 2026 International AI Safety Report does not say that current AI systems are capable of seizing control from humans. It says current systems lack the capabilities necessary for a genuine loss-of-control scenario, while also documenting continued progress in capabilities relevant to such scenarios. Researchers remain deeply divided over how likely future loss of control is. 2026 International AI Safety Report

That is a far more useful conclusion than either “we are doomed” or “there is nothing to worry about.” Current systems are not the hypothetical superintelligence at the center of the extinction debate, yet systems are becoming more autonomous, more capable of carrying out extended tasks and better able to operate within complex technical environments. The practical warning, then, is not that an AI system is secretly plotting our destruction. It is that the relationship between capability and control is becoming more consequential, and organizations may be granting systems meaningful operational reach before they have equally mature ways to contain that reach.

Anthropic’s cybersecurity evaluations offer a useful example. In July, the company reported incidents in which Claude models reached the live internet from evaluation environments and gained unauthorized access to real systems. In one case, the model accessed a database containing several hundred rows of production data. In another, a model published a malicious Python package to PyPI; the package was available for roughly an hour and was downloaded and executed on 15 real systems before it was removed. Anthropic cautioned that these were isolated evaluation incidents rather than controlled experimental evidence, and that the models had been assigned offensive cybersecurity tasks. Anthropic’s cybersecurity incident analysis

Those qualifications matter. These systems were not independently deciding to attack organizations for their own purposes, and the incidents should not be repackaged as evidence of an AI “escaping.” But dismissing them because they occurred during evaluations would miss the operational lesson. Capability, access, environmental configuration and assigned objectives combined in ways that produced unintended real-world consequences. Several assumptions that were supposed to hold across the environment did not hold. That looks much less like science fiction than like a familiar systems failure.

The timing problem in AI governance

Most governance programs still operate at human speed. A system is approved, ownership is assigned, risks are documented, controls are established and someone monitors performance. When an issue appears, it is investigated and escalated. The appropriate authority makes a decision, and someone implements it. There is nothing inherently wrong with that sequence. The problem is the clock.

AI agents increasingly challenge that timing model. Systems can operate computer interfaces, collaborate with people and other systems, and carry out increasingly extended work. OpenAI Chief Scientist Jakub Pachocki has described current reasoning systems as already capable of operating computers and graphical interfaces, collaborating with humans and other systems and carrying out research projects, while warning that alignment and monitoring remain significant concerns as capabilities advance. Jakub Pachocki, “An Alien Mind”

The organization surrounding those systems may still operate very differently. An alert is generated. Someone opens a ticket. The ticket is triaged. The issue moves through an escalation chain. People exchange messages, schedule a discussion, obtain approval and eventually make a technical change. The mismatch is not merely procedural; it is temporal. I think of that gap as governance latency: The time between recognizing that governance needs to act and making that decision real in the system. Governance is effective only when it can become operational before the consequence it is intended to prevent or contain.

For leaders, that changes the conversation. The question is no longer simply whether an AI system has an owner. Can that owner actually revoke the system’s authority, and how quickly? Can the organization determine which systems the agent has touched before it touches another one? If a model begins behaving outside approved operating conditions, does human approval become an actual execution gate, or does the requirement exist only in policy? Can investigators later reconstruct what the system knew, which tools it used, which data informed it and which human interventions affected the outcome? If governance decides at 10:02 that an action is no longer permitted, what time does the system become technically incapable of performing it? A five-minute answer and a five-day answer may describe the same policy, but they do not describe the same governance capability.

Accountability without authority is a paper control

AI governance conversations often settle comfortably on ownership. Assign a model owner, identify a business owner, name the risk owner, document the system owner and now someone is accountable. That is necessary, but it is not control. A person can be accountable for an AI system and still lack the technical authority to constrain it. A governance committee can prohibit an activity while the production architecture continues to permit it. A risk register can correctly identify an unacceptable condition while doing nothing to prevent that condition from becoming operational reality.

This is the point where governance stops being primarily a policy exercise and starts becoming an engineering requirement. NIST’s AI Risk Management Framework already provides some of the scaffolding for that view. Its Core describes governance as a cross-cutting function and states that risk management should be continuous, timely and performed throughout the AI lifecycle. NIST AI RMF Core

The unresolved question is what happens between a governance decision and an operational state. A policy that says an agent may not access a particular class of data eventually has to become a permission boundary. A requirement for human approval has to become an actual execution gate. An obligation to maintain traceability has to become logs and provenance that remain available long enough to investigate. An escalation rule has to connect to someone who has both the authority and the technical means to stop what is happening. Rollback, credential revocation, execution limits, authentication requirements, network boundaries, provenance, shutdown mechanisms and decision logging may sound like engineering details, but they are also what governance looks like when it reaches production.

Without that translation, an organization can have a sophisticated AI governance program while remaining surprisingly unable to govern the AI system at the moment governance matters most.

We are using the wrong threshold for concern

The research community should continue studying existential risk. It would be reckless to dismiss a low-certainty risk simply because its consequences are difficult to quantify, particularly when the potential loss is enormous. But executives do not need to know the probability of human extinction before deciding whether their own AI systems are controllable.

Imagine autonomous or semi-autonomous AI operating in cybersecurity, financial services, healthcare, logistics, critical infrastructure, intelligence or defense. None of those systems needs consciousness, hostility toward humans, a survival instinct or a secret agenda. A system needs only enough capability, access and authority to produce consequences faster than the organization surrounding it can understand what is happening and make an effective intervention.

That organization may still have an AI governance council. It may have risk registers, policies, responsible executives, model cards, review boards and named owners. On paper, the system is governed. Operationally, the more important question is whether the governance can still reach the system in time.

That is what the extinction debate risks obscuring. The immediate challenge is not proving that AI will someday become uncontrollable. It is determining whether we are already building systems whose operational reach is beginning to exceed the operational reach of the governance surrounding them. If that gap keeps widening, we will not need an extinction event to learn that our governance model failed. An ordinary incident will be enough.

The conversation around AI agents has shifted remarkably fast. A year ago, I was mostly worried about what an AI system might say. Now I am increasingly worried about what it can do.

That distinction changes my security model.

An agent that gives me a bad recommendation creates a problem I may still have time to catch. An agent with credentials and tools can turn the same bad decision into an API call, a privilege change, a deleted resource or an external transaction before anyone realizes what happened.

I saw a particularly uncomfortable example in OWASP’s GenAI exploit roundup: an AI agent connected to a real inbox began deleting messages and reportedly continued despite attempts to stop it. There was no sophisticated attacker required. The combination of live access, destructive capability and insufficient confirmation was enough.

That is why I don’t think “human in the loop” is specific enough anymore.

I want to know exactly where the human — or another independent authorization control — is in the loop.

I would let agents autonomously perform thousands of low-risk actions. But there are five actions where I would deliberately put another decision between the agent’s intent and execution.

1. Move money

I am comfortable letting an agent analyze invoices, reconcile accounts, identify anomalies, prepare a transaction and recommend that a payment be made. I would not let the same agent unilaterally complete a consequential financial transaction.

The reason isn’t simply fraud. The agent could be operating on manipulated data, an indirect prompt injection, incorrect business context or a perfectly legitimate instruction that it interpreted incorrectly.

The approval request should therefore expose the transaction itself: Pay $X → recipient Y → from account Z → because of instruction A.

The second approver should be authorizing that exact transaction, not approving a vague statement such as “complete the payment workflow.”

This is one place where I think we need to borrow more aggressively from financial controls. The person — or system — that prepares a consequential transaction should not also be the final authority that releases it.

The same principle should apply to an agent.

2. Delete production data

This is the easiest line for me to draw.

I want agents helping operations teams identify stale resources, clean environments, investigate incidents and recommend remediation. I don’t want an agent translating “clean this up” into an irreversible production action without another control validating what “this” actually means.

OWASP describes exactly this class of failure in its guidance on Excessive Agency: an AI system can become dangerous when excessive functionality, permissions or autonomy allow it to perform damaging actions. OWASP specifically recommends human approval for high-impact actions.

For a destructive action, my approval screen should tell me:

What will be deleted? What depends on it? What is the blast radius? Can I recover it?

“Delete these 12 identified resources after these dependency checks” is reviewable.

“Clean up unnecessary production resources” is not.

The difference matters because I don’t want a human approving the agent’s interpretation. I want the human approving the actual action.

3. Change privileges

This is where I become even more conservative.

Imagine an agent reaches a point in a workflow where it cannot continue because it lacks permission. It decides the permission is necessary, requests additional access and then has a path to grant or activate that access itself.

At that point, the agent is no longer merely executing within an authorization boundary. It is participating in defining its own boundary.

I would separate those decisions.

An agent can tell me it needs more authority. It cannot be the authority that decides whether it gets it.

That means changes to IAM roles, privileged group membership, credentials, security policies and sensitive tool permissions need an independent authorization path.

This is also why least privilege becomes more important, not less, with agents. OWASP recommends that agent tools operate with the minimum functionality and permissions necessary and that downstream systems enforce authorization rather than trusting the model to decide whether an action should be permitted.

A compromised read-only agent gives me an incident. A compromised agent that can make itself an administrator gives me a very different incident.

4. Expose sensitive information

This one is deceptively difficult because an agent doesn’t need an “exfiltrate data” tool to cause a disclosure.

It may attach a document to an email. Populate a support ticket. Paste source code into an external service. Send customer information through an API. Give another agent information that the second agent was never supposed to receive.

Each step can look legitimate.

The EchoLeak vulnerability in Microsoft 365 Copilot demonstrated why this boundary matters. A specially crafted email could influence the AI system and create a path toward disclosure of sensitive organizational information without the user needing to click a malicious link or attachment.

Before sensitive information crosses an established trust boundary, I want another decision point that answers four questions:

What data is leaving? Who is receiving it? Why do they need it? Where is it going?

This is where approval design matters almost as much as approval itself. If the agent controls the description shown to the approver, “Send requested information to vendor” might conceal a very different underlying action.

The security control therefore needs to derive its approval context from the actual transaction — destination, data classification, tool call and requested scope — not merely from the agent’s natural-language explanation.

Otherwise, we have put a human in the loop without giving that human enough information to make a security decision.

5. Create or delegate privileged access

This is the boundary I expect security teams to wrestle with most as multi-agent architectures become common.

A modern agent workflow can quickly become:

User → Agent → Delegated Agent → Skill → Tool → Resource

The original agent may have legitimate authority. But what happens when it delegates a task to another agent that invokes a tool holding broader credentials?

The question I care about is not simply whether every component is authenticated.

It is whether the authority exercised at the end of that chain still represents what was authorized at the beginning.

I would therefore require another authorization decision when an agent creates privileged access or materially expands or delegates consequential authority.

And I would make that approval very specific:

What authority is being delegated? To whom? For what task? For how long? Can it be delegated again? What happens to it when the task ends or the original authorization is revoked?

OWASP’s AI Agent Security Cheat Sheet addresses excessive autonomy, tool abuse, privilege boundaries, sensitive-data exposure and multi-agent risks. Those risks become harder to reason about as an agent’s authority travels through more components.

Authentication tells me which agent is acting. Authorization needs to tell me whether this particular action, with this particular authority, is still allowed.

Those are not the same question.

A second approval doesn’t always mean a second person

There is one part of this argument I don’t want misunderstood.

I am not proposing that enterprises hire armies of people to click “Approve” every time an agent wants to do something.

That would destroy the value of automation.

For the highest-risk actions, a human approval may absolutely be appropriate. In other cases, the second decision could come from an independent policy engine, authorization service, transaction-control system or another deterministic security control that the initiating agent cannot modify or bypass.

The NIST AI Risk Management Framework takes a risk-based approach to managing AI risks and emphasizes governance throughout the AI lifecycle.

That’s how I think about agent autonomy as well.

The question isn’t: Should this agent be autonomous?

The better question is: Which actions can this agent perform autonomously, and which actions require an independent decision?

For me, the dividing line is increasingly clear.

Let agents search, analyze, summarize, correlate, recommend and prepare. Let them autonomously execute actions that are bounded, observable, reversible and already inside clearly established authority.

Add friction when an action can move money, destroy production data, increase privilege, expose sensitive information or create authority that propagates beyond the original task.

Most importantly, I would never design a workflow where the same agent can propose a consequential action, acquire the authority necessary to perform it, approve that authority and execute the action.

That isn’t autonomy I can govern. It’s self-authorization.

And when AI agents can operate at machine speed across multiple connected systems, a few seconds of independent authorization may be some of the cheapest incident prevention we can buy.

Twenty-two rows. That was the AI inventory in a board pack I read last year, and it was a good one. Every system named, every owner listed, every risk rating filled in and colour-coded. Somebody had worked hard on it, and the committee approved it in four minutes.

I kept looking for a column that wasn’t there. The pack could tell you who owned each system. It could not tell you who was allowed to say no to one.

Those sound like the same thing right up until the morning they aren’t.

1. Signal. AI has moved inside the operating model

When a director asked me last spring what our AI risk was, I gave her the honest answer. Which was that I no longer knew, because the question had changed shape while everybody was still busy answering the old one.

Twelve months earlier I could have told her precisely. We had a model inventory, a review process and a register entry saying the right things about bias and data quality. All of it was true, and all of it had quietly stopped being the point, because the systems had moved inside the work. They chose which cases went to which queue, which prices moved and which alerts a human ever saw.

Foundry’s State of the CIO 2026 report puts the shift plainly. Roughly three-quarters of leaders say AI is already reshaping how their operations run, and the number climbs higher in financial services. Around seven in 10 expect deeper involvement in agentic systems this year.

Don’t let those figures become the story. Adoption numbers are the easiest thing to produce and the least useful thing to govern from. Your board can watch that percentage climb for three years and learn nothing it can act on. The signal underneath is simpler. AI has stopped helping people finish tasks and started shaping how work gets allocated, how rules get applied and what the company does next.

Which means your board needs a better question than how much AI do we have. It needs to know which decisions have changed hands.

2. Consequence. Authority travels faster than accountability

Accountability used to be visible. Person, role, authority, decision. You could follow it with a finger, and when something went wrong you could find the finger.

That chain now reads: human, model, agent, system, supplier, outcome. Every link has an owner and a budget line. The decision itself often has neither.

Watch it happen. A team buys a tool to help underwriters. It starts recommending, and it’s good, so people gradually stop disagreeing. Six months later somebody switches on auto-approval below a threshold, because the review queue is a bottleneck and the numbers support it. Nothing in that story needed a bad actor. Authority moved three times, and nobody recorded any of it.

That’s board risk, not model risk. Call it authority drift. Real decision-making power moves outward while formal accountability sits exactly where somebody wrote it down in 2023.

The same study found most organisations have or plan an AI steering group, with IT sitting on the great majority of them. Only about half report a formal approval process. Structures arrive fast, because they’re easy to announce and cost nothing but a recurring meeting. Decision rights arrive slowly, because somebody has to accept them.

I once watched a company stand up three AI committees inside a year and still be unable to name one person who could stop a model.

3. Decision. Who gave the machine permission?

Your board can’t govern every AI action and shouldn’t try. Trying to do so creates a register nobody reads and a comfortable sense that the ground is covered. Insist instead that management names the material decisions.

Material means it moves customers, capital, employees, regulatory standing or resilience. The tool that writes meeting notes doesn’t qualify, no matter what the vendor promised. A system shaping credit, pricing, workforce reductions, claims handling or cyber response does. Most firms land on 15 or 20. If management returns with two, they haven’t looked. If they return with three hundred, they’ve listed systems rather than decisions.

Then, for each one, four verbs.

• Recommend. The system proposes, and a person chooses.
• Decide. The system chooses, inside limits somebody set in advance.
• Execute. The system chooses and acts, with no human in the transaction.
• Accept. Somebody carries the consequence when it goes wrong.

That last verb is where rooms go quiet, and it’s the only one that can’t be handed to a machine.

Put them together, and you’ve drawn the authority envelope. What may this system recommend, decide and execute? Where does a person pick the thread back up? Whose name sits on the outcome? It’s a dull artefact, half a page per decision, and it’s what your board should ask to see instead of the AI policy, which everyone has, and nobody reads.

4. Evidence. Show me where the authority stops

An envelope on paper is a claim. Then you ask what proves it.

Start with the system itself. If AI may approve up to fifty thousand, is fifty thousand built into the workflow or typed into a policy? Only one of those holds at three in the morning, when the queue is long, and somebody is under pressure.

Ask which exceptions have been granted. Who approved each, who accepted the risk left behind and when it expires. An exception without an expiry date quietly becomes the operating model, and eighteen months later somebody calls it legacy and asks for budget to remove it. Read the 10 oldest entries in that register, and you’ll learn more than the policy will ever tell you.

Ask how many important decisions ride on the same model, the same cloud, the same supplier. Firms with 20 AI applications regularly discover they have two real dependencies, because separate teams bought separate products built on the same substrate. Procurement records won’t show it. The architecture diagrams and the person who pays the bills will.

Ask whether the business can still decide if the service vanishes. Not whether IT can restore it, which is a different and easier question. Who does that work by hand, how long can you last without it and has anybody currently employed ever done it?

Ask what returns authority to a person: confidence dropping below tolerance, customer harm, a breach of appetite. Then look for one more that most firms lack, because their triggers came from incident management, where failures announce themselves. A drifting model doesn’t crash. It carries on answering plausibly, wrong by small margins, at volume, logging cleanly.

Finally, ask how long it takes to reconstruct one decision from end to end. An afternoon is evidence. A fortnight is archaeology.

5. Challenge. Follow the decision, not the inventory

Five questions. Directors don’t need to become engineers to ask them, and management can’t deflect them with architecture diagrams.

Which material decisions now depend on AI? A vague answer means nobody has drawn the list, which is itself the finding.

Who owns the authority for each one? Not the model owner, not the committee chair. If the answer names a team, you’ve been told who keeps the system running, not who let it decide.

How far has that authority travelled: recommend, decide or execute? Firms are usually one step further along than their paperwork claims.

What brings it back to a person? If the answer is human oversight, ask what forces the human to look while everything still appears fine.

What evidence shows the system stayed inside its limits, and how quickly can you produce it?

Answer all five for the top 10 decisions, and you’re in decent shape. Answer four and go quiet on the fifth, and you’ve found where the work is. The quiet is the useful part.

6. Horizon. Today’s approval expires quietly

The risk worth watching is expansion without reauthorisation, not a dramatic failure.

A provider swaps the underlying model in a routine release. An agent picks up new permissions because somebody connected it to another system. The business points the same tool at a bigger question, since it worked on the small one. More sensitive data joins the workflow. Reliance grows, so the review that was thorough in month one becomes a glance by month nine.

Nothing breaks. Nothing gets announced. The approval granted a year and a half ago now covers something it was never asked about.

Agentic systems will sharpen all of this, because their entire value lies in doing more without being asked each time. That’s their point, and it is also the governance problem stated in one sentence.

So the question I’d put on the agenda isn’t whether you’re monitoring continuously. It’s this. What change would make today’s approval no longer good enough? Write that answer down while everyone is calm, and you have a trigger instead of a hope.

Risk doesn’t only rise when the model changes. It rises when you quietly ask the same model to decide something that matters more.

7. Action. Map the decisions before they map themselves

One deliverable, not a framework. Most firms already have a framework, and it hasn’t helped.

Ask management for a map of the top 10–20 material AI-supported decisions. One row each. The business owner. What the AI actually does, in the four verbs. The authority limit, as a number. Who approved any exception and when it expires. The main dependency. The fallback, and who performs it. What forces escalation. Where the evidence lives.

Nine columns, two pages. It will take longer to produce than anybody expects. That delay tells you something before you read a word, because a company that fills it in a fortnight is already governing this and one that needs a quarter is still discovering it.

What comes back will show you orphaned decisions, ambiguous authority, stale approvals, exceptions nobody closed, concentration hiding behind a healthy supplier count, and a fallback that exists only on paper.

Boards don’t need to own AI. They do need to know who owns the decisions AI is now allowed to make.

AI can inherit authority. It cannot inherit accountability.

At a Fortune 500 insurer, I watched an AI agent inside the month-end close do exactly what we asked. It posted the recurring accruals. It worked the intercompany exception queue, the reconciliation that normally falls to a staff accountant. And when items would not match, it cleared them to a suspense account within the configured tolerance rather than escalating them to a human. Clearing the queue was the objective. Nobody had defined escalation as success.

Every journal entry carried the controller’s user ID. No one set out to obscure anything. The job step had been mapped to her credentials at implementation, which is how it had been done since the system went live.

Segregation of duties had become a fiction. The control requires a named preparer and a named approver. The agent did both, under one identity: it prepared the entries and it disposed of the exceptions. The record named the controller for each. She had done neither. When internal audit walked those entries the following quarter, they asked her to explain postings she had never seen.

None of this was a model failure. The agent was accurate. It was authorized. Every entry looked clean, which is exactly why nobody caught it.

Vipin Jain

Here is the number that should frame this year. Gartner expects 40 percent of enterprise applications to carry task-specific AI agents by the end of 2026, up from under 5 percent in 2025. Okta finds only 10 percent of organizations have a strategy for managing the agents they are already running. And Deloitte, surveying more than 3,000 leaders across 24 countries, puts mature agentic governance at 21 percent against 74 percent who expect to be using agents within two years.

Adoption is racing. The control model is not. Every one of those agents inherits a permission model built for a person clicking a button, which means the question that matters is no longer whether an agent can do the work. It is whether, when it does, anyone can say who authorized it. McKinsey frames the same shift from the governance side: organizations can no longer concern themselves only with systems that say the wrong thing. They now have to contend with systems that do the wrong thing.

What actually broke?

ERP, CRM and core administration platforms rest on four assumptions so foundational that nobody writes them down: identity, permission, action, receipt. Each one places a named human at the center. An agent breaks all four at once, and none of them repairs itself.

Vipin Jain

Intent. A person submits a bounded request, and the request is the transaction boundary. An agent receives a goal and decomposes it into steps at runtime. On a state eligibility modernization program, we gave an agent one goal: clear the verification backlog. It re-ran electronic verification, and where a federal source returned no match, it advanced the case toward closure and generated the notice. Nobody listed that step. Nobody had to. It is a defensible reading of clear the backlog, and it is precisely what the agent was rewarded for. But a no-match is not a finding of ineligibility. It triggers a reasonable opportunity period, and closure carries notice and appeal rights that attach to a determination, not to a data lookup. The rule that breaks here is least privilege. You cannot scope permissions to an action set nobody enumerated in advance.

Authority. Applications grant entitlement to a person. It is standing and role-based, and that person’s judgment is the unwritten limiter on it. At a specialty retailer, I watched a quoting agent go into Salesforce to compress turnaround at quarter close. The approval matrix was intact: manager above fifteen percent, director above twenty-five. None of it fired. A submission action in the user interface triggers a Salesforce approval process, and the agent wrote to the record through the API. It never submitted. An integration user provisioned with more rights than anyone intended bypassed the validation rules and record locks that would ordinarily catch it.

Salesforce’s own architects are blunt about the mechanism. The client credentials flow, they warn, creates a service-account pattern in which every call runs as a single identity, and both per-user permissions and the audit trail are lost along the way. Their guidance on identity propagation goes further: when an agent calls a downstream service using client credentials, the request carries the agent’s service identity rather than the end user’s, which makes per-user authorization and audit trails impossible.

That approval process was never a security control. It was a workflow convention that humans had no reason to route around.

Logic. Business rules used to be a configuration artifact: versioned, testable, under change control. At a health insurer, I traced one authorization threshold for advanced imaging to three separate homes. Configured in the core administration platform. Routed in the orchestration layer. And written in plain English inside the agent’s instruction, as a sentence somebody added during a sprint. Medical policy tightened the first. Nobody touched the third, because the third is not a configuration item. No change control that clinical policy or compliance has ever seen covers it.

In every payer program I have worked, that drift runs one direction only. An agent cannot lawfully deny since adverse determinations require review by a licensed clinician. So the exposure is silent over-authorization against a threshold the organization believes it already tightened, surfacing months later through a delegated vendor audit or a utilization variance that finance notices before medical management does. Then someone asks the question with no good answer: how long, and how many.

Evidence. The log names whichever credential was presented. That is the insurer’s close, and it is the most measurable of the four failures. The Cloud Security Alliance found that 68 percent of organizations cannot clearly distinguish between human and AI agent activity, even as 73 percent expect agents to become vital to operations within a year.

An agent without a delegation chain is not automation. It is an unsigned transaction.

Table 1. The same failure, four industries

Why didn’t the controls catch it?

Because in every one of these cases, the controls worked as designed. That is the uncomfortable part, and in the programs I advise it is the finding that changes the conversation. This is not a problem the CIO can hand to the security team and consider closed.

Consider the insurer’s segregation-of-duties conflict. It never appeared in the access risk analysis, and the tooling did not fail. Most implementations exclude technical and service accounts from risk-analysis scope entirely, on the reasonable historical assumption that a service account does not exercise judgment. Nobody ever analyzed the account. The conflict was not missed. It was never in view.

Most writing on agent risk assumes an adversary. The OWASP Top 10 for Agentic Applications, built with input from more than 100 practitioners, catalogs goal hijack and memory poisoning and places Identity and Privilege Abuse third on the list. What I see most often is duller. Nobody attacked the payer. A threshold governed in one of three places simply drifted, and it took a vendor audit to notice.

The pattern holds across all four, and each one breaks a control an auditor already asks about. Intent breaks least privilege. Authority breaks separation of duties. Logic breaks configuration management. Evidence breaks non-repudiation. Those four are the spine of any controls walkthrough, which is precisely why this arrives on the CIO’s desk rather than staying in the security function.

None of these produced a headline. They produced audit findings and rework: a control deficiency to disclose, authorizations to re-adjudicate, cases to reprocess, notices to reissue. That cost lands on the CIO’s budget and the CFO’s certification, not on the security team’s incident log.

The repair is the same in all four cases. Stop letting an agent borrow a person’s credential, and start recording who stands behind it.

Vipin Jain

What has the market already conceded?

What I find most striking this year is not that a vendor shipped something. It is that vendors, standards bodies, regulators and advisers all converged on the same requirement within roughly six months of each other, without coordinating.

Vipin Jain

Microsoft made the human sponsor mandatory. Every agent identity carries a named business owner accountable for it, and if that person leaves, sponsorship transfers to their manager so accountability never quietly disappears. SAP now governs an agent lifecycle that runs from proposed to evaluated to approved to active to retired, with a verification badge controlling which agents are cleared for use. And Okta has moved access decisions out of individual applications and into the identity layer, with AWS, Google Cloud and Salesforce behind the effort.

Google reached general availability this month with the bluntest statement any vendor has made. Its agent identity documentation argues against the service account directly: agent identities are not shared, cannot be impersonated, and permit no long-lived keys. Every action resolves to a cryptographic ID rather than to whoever the credential once belonged to. That is the insurer’s close, described by a hyperscaler as a design flaw to engineer out.

The pattern held into this month. At Black Hat on 4 August, Rubrik unveiled a service that grants an agent access one tool call at a time, minting a short-lived token for each call and eliminating standing permissions outright. Its general manager for AI put the diagnosis in one line: the access models we built for humans were never designed for autonomous actors.

The standards work is further behind, and honest about it. NIST’s National Cybersecurity Center of Excellence opened a concept paper on software and AI agent identity and authorization in February and closed comment in April, explicitly seeking input on agent auditing, non-repudiation and prompt-injection controls. The gap is officially acknowledged. The standard is not written.

Regulators moved in a way most leaders misread. Brussels deferred the EU AI Act’s high-risk obligations in June, pushing them to 2027 and 2028, and the headline everyone absorbed was that the AI Act had been delayed. Half true, and imprecise in a way that matters: the Article 50 transparency duties, including telling people when they are interacting with an AI system, were untouched. They took effect on 2 August and are live now.

The advisory community has landed in the same place. BCG, writing on scaling agents in regulated industries, tells CIOs to define clear approaches to agent identity, authority and accountability: what agents can access, what actions they can take, and how those actions remain traceable. Booz Allen, surveying 105 federal IT and cybersecurity decision makers this spring, found 22 percent who have not determined who bears responsibility when an agent causes an incident.

Read those together and the picture is unambiguous. The identity layer has been rebuilt around delegated, sponsored, per-action authority. The business applications those agents act inside have not moved at all.

The plumbing now encodes accountability. The applications still do not ask for it.

Where should CIOs start?

None of this requires a reorganization. The leaders I see getting ahead are not doing more; they are doing six things in order, on the bounded use cases where they can afford to learn.

1. Inventory the agents. Enumerate every agent acting inside a system of record, the identity it runs under, and its standing entitlements. Only 23 percent of leaders report full visibility into the agents already running in their environment, and most organizations I work with cannot produce that list today, and producing it is usually the moment this becomes real to the executive team.
2. Bind identity to a named human. Retire shared service accounts and borrowed logins. Every agent gets its own identity with a named sponsor who answers for it. This is the change that fixes the insurer’s close because the document header stops naming a bystander.
3. Scope authority per action. Short-lived, time-bound, just-in-time. And move approval enforcement out of the workflow layer into the authorization layer, where an API call cannot walk around it. This is the step I almost never see taken, and it is the one that fixes the retailer.
4. Consolidate the logic. The instruction becomes a configuration item: versioned, evaluated, approved and reviewed alongside the rules engine. If a threshold changes in medical policy, it changes in all three places or it changes in none.
5. Instrument the evidence. The transaction record carries the delegation chain, meaning agent, sponsor, instruction version and model version. Without those four, every incident investigation starts from zero.
6. Graduate autonomy deliberately. Sanctioned, then supervised, then autonomous, with sunset as a real state rather than a euphemism for neglect.

Vipin Jain

The gates between rungs are what make the ladder operational rather than decorative. Sanctioned to supervised requires a named sponsor and a declared scope. Supervised to autonomous requires an evidence record showing the agent stayed inside it. And sunset is not a failure state. It is what happens when an agent stops earning the authority it holds, which most organizations have no process to detect.

Do these in order and autonomy compounds. Skip them and you join the more than 40 percent of agentic projects Gartner expects to be canceled by the end of 2027, undone not by weak models but by escalating cost, unclear value and inadequate risk controls.

The takeaway

The most common strategic mistake I see this year does not look like a mistake. Leaders are granting agents authority inside systems whose architects assumed a human held it, and then treating the resulting mess as a technology problem. It is not. It is the same failure I have watched for thirty years, from trading floors to Medicaid programs: business strategy, IT and organizational culture are not architected to move together, and the technology simply industrializes the gap.

I recommend a reframe most leaders resist at first. Guardrails here are not friction. You can give an agent that proves who authorized it more autonomy, sooner, and with far less anxiety than one that cannot. The delegation chain is what makes acceleration survivable.

So put three questions to your team this week. None of them requires a program to answer, and all three will tell you more than a maturity assessment.

Table 2. Three questions, and what a good answer sounds like

The agent is the new user. Your applications have not been told.

This article was made possible by our partnership with the IASA Chief Architect Forum. The CAF’s purpose is to test, challenge and support the art and science of Business Technology Architecture and its evolution over time as well as grow the influence and leadership of chief architects both inside and outside the profession. The CAF is a leadership community of the IASA, the leading non-profit professional association for business technology architects. 

AI adoption in the enterprise is way up — and so too are IT leaders, into the night, dealing with the risks and governance issues of AI use.

To be sure, increasing attention to AI governance is a welcome addition to enterprise AI strategies, shifting an anything-goes approach toward risk-aware plans of action directed toward value. But it’s also taking an unspoken toll on those responsible — with the challenges of agentic AI mounting fast.

Not only are four in five senior business decision-makers, including CIOs, CISOs, and CDOs, using more of their day to manage AI risk, but their working hours are up 26% on average due to the issue, according to a new survey released by AI governance platform vendor OneTrust.

Contrast that to recent findings from Boston Consulting Group, which found that 42% of frontline employees who regularly use AI save nearly a full day of work each week, the majority of whom are given no guidance on what to do with that time saved, and you get an organizational picture of rampant activity, questionable direction, and lots of high-salaried time spent on cleanup and oversight.

Much of leaders’ extra time spent on managing AI risk is due to an increased awareness of AI risk and governance, IT leaders and industry observers say, but triage and other factors play a significant role as well.

For example, a third of respondents to OneTrust’s survey say they’ve seen employees use unapproved AI because approved tools or processes weren’t available quickly enough. Eighty-six percent have reported AI-related incidents, and over a quarter have experienced two or more incidents of AI systems taking unapproved actions.

However, as Blake Brannon, chief innovation officer at OneTrust, sees it, a huge part of the equation is AI adoption, with use skyrocketing.

“You’ve got the sheer volume and adoption of AI in organizations,” he says. “Whatever AI they were using at the beginning of the year, it’s probably more than 2x what they’re now looking at using inside the company, and that’s a compounding thing.”

Governing citizen coders and unsanctioned use

One key factor is increasing permission for non-IT employees to use AI to build their own apps, Brannon says. That move toward citizen development sparks a greater need for governance, he adds.

“We’re seeing exponential growth, not necessarily in different AI providers or tools, although that is growing, but the explosion of citizen builders and the new creativity that has been unlocked,” he says.

AI is creating new challenges for IT leaders as fast as they can understand them, he says, and the technology complicates risk management because of the speed at which it can make decisions and create problems.

“Security, compliance, risk management historically have just had to govern humans,” he says. “Humans were deterministic, and they moved at human speed.”

Viren Meghani, technical architect at Tata Consultancy Services, also sees IT leaders spending more time on AI governance and risk management than a year ago, but believes that’s largely related to shadow AI.

“Interestingly, this increase isn’t coming from overseeing the approved tools,” he says. “Instead, it’s all about chasing down the unapproved ones.”

Meghani believes employees’ urgency for AI solutions is a key factor. “People don’t wait around; they find a tool that works for them,” he says of unsanctioned AI use. “Then, governance has to scramble to put controls in place around something that was never properly evaluated.”

Spending more time on governance also doesn’t mean IT leaders are doing it correctly, he adds.

“Time spent reacting to incidents doesn’t equate to time spent on building effective controls,” he says. “If you’re busy investigating what an unapproved tool did, you’re not really governing; you’re just putting out fires.”

The organizations taking AI governance seriously invest time in architecture before deployment, using methods such as data lineage, model versioning, escalation paths, and audit trails, Meghani explains. “That kind of work might not grab headlines, but it’s what really helps to lower the number of incidents,” he adds.

Awareness isn’t readiness

Conal Gallagher, CIO and CISO at IT management software vendor Flexera, also believes taking governance seriously doesn’t mean organizations have achieved AI readiness.

“The risk surface of AI has expanded so quickly that the time designated to managing that risk continues to increase,” he notes. “Just one year ago, much of the focus was on enabling employees to experiment with AI, but now we’re seeing cases where autonomous agents create other agents and identify vulnerabilities at incredible speed.”

Governance is constantly playing catch-up to the changes happening with AI, Gallagher adds.

“Our IT team’s approach is to be judicious about which capabilities we put into employees’ hands and to slow deployment when we don’t understand the risk well enough,” he says. “We’re also applying established security principles to make sure agents don’t have greater access to capabilities than they actually need. The technology may be new, but some of the most important controls are still familiar ones.”

Anant Adya, EVP and head of Americas delivery at Infosys, says he’s spending more time managing AI risks now than a year ago, a natural shift, he adds, given the company’s move from experimentation to enterprise-scale adoption.

“While it does require additional attention, it is becoming embedded into our operating model rather than simply adding hours to the day,” he says of Infosys’ added emphasis on governance, security, privacy, compliance, and responsible AI practices.

That effort to operationalize governance and control is vital as the technology moves into core business workflows, Adya says, especially as agentic AI — what’s next — will only add another level of urgency.

“As these systems gain greater autonomy, they can access sensitive data, interact with other systems, and take actions on behalf of an organization,” he says. “Enterprises need strong identity controls, continuous monitoring, and appropriate human oversight.”

The struggle for good governance remains an issue for many organizations, but CIOs facing an overwork problem playing catch-up on AI governance are advised to get a handle on enforcing what matters before autonomous workflows compound the challenge.

My team was already using AI when I joined the company a year ago, and I quickly spotted a bottleneck. We’d finish large product requirements documents that then sat in inboxes for a day or two before someone read them and handed the work to an agent. To cut the cycle time, I had the recipient’s agent do the pre-read instead, sending questions back to the authoring agent as needed.

I realized we would see even more efficiencies if the agents interacted with each other the way human teams do. So, I stopped asking people to use AI to do their own jobs faster, and started asking them to hire and manage agents instead, like junior employees. They train them, set detailed expectations of outcomes, review plans, run periodic checks, make sure they collaborate with peer agents and own the quality of the output. A PwC survey of senior executives found the same thing: organizations adopting agents report gains, but the value concentrates where agents work across functions rather than in isolation.

Seven people on my team each work with a role-based primary agent, backed by subagents for specialized tasks, and I use agents for all of my functions. They’re full participants in the software development life cycle, not prototypes. Each person owns their agent budget and evaluates new tools for our stack. For example, the product management agents triage incoming customer requests, research and define requirements, and collaborate with peer agents.

Muffin, the product design agent, works in two modes: designing the frontend experience for a feature in the sprint, or exploring ideas not yet on the roadmap. In the background a separate agent responds to Slack alerts, triages issues and queues work for the project-manager and developer agents, pinging the person only when they are needed.

Muffin works with Travis, the frontend engineering agent, and delegates to her own subagents. A design system agent named Ive checks our components before anything new is built. Sagmeister reviews visual design and accessibility, and Dieter digs through old branches for context. A designer hands Muffin Figma mockups, and her team turns them into working code. The code goes to Travis, who runs it past his own subagents, so it meets our frontend engineering standards.

PacMan, an agent our research team built, sits on top as the orchestrator. It takes a ticket from Waffle, the product management agent, and routes the work: engineering to Travis, design to Muffin and questions about our proxy technology to Jarvis. A human signs off before anything moves forward. PacMan coordinates across product management, design, frontend and backend, and keeps the humans moving — maybe better than we managed on our own. “It’s unnerving that the agents are now driving the humans rather than the reverse,” says Jim Beno, our head of product design.

The moment I knew we had crossed a threshold came with a request for a new AI FinOps dashboard to show all the tokens consumed by each team, correlated to the work each team had done. Humans and agents worked in one room, and we went from a whiteboard concept to a functional prototype running live against real AI traffic in just one hour.

Agents earn trust the way junior hires do

Getting there took more time than I expected, and it required a lot of micromanaging early on. Agents graduate to bigger jobs as they prove themselves. Sometimes I handed one something too big, checked back 24 hours later, and the result was garbage. Over time they took on more complex work with less direction. After each project, I walked the agent through a retrospective, teaching it to notice what it had learned and fold those learnings back into its memory. Many agents now have more native memory, but ours was homegrown at the start.

When Muffin started delivering designs in code, they looked right on the surface but used the wrong components and missed our branding. Developers had to rework half the output, creating tension between teams. To resolve this, Alan Dong, the engineer who built Travis, loaded frontend best practices into its knowledge base. Separately, we changed how we prompt: treating the agents like junior hires, giving them more context than feels necessary, and debriefing on lessons after each session. We built a workflow where Muffin creates the designs in code and Travis reviews them. That collaboration got us to roughly 95% acceptance from the frontend team.

Another time, a wording mix-up nearly triggered a premature build. Katie Chen, senior product designer, selected “finish the load animations” from a list of options, just to point Muffin at the task. Muffin took it as a green light and started building. Katie stopped her mid-stream. Now Muffin asks clarifying questions first, and nothing gets built without an explicit go-ahead.

Early agent adopters moved so fast that the rest of the organization couldn’t keep pace. One person on the team built a full-featured Model Context Protocol (MCP) catalog that sat in engineering for months before anyone built on it. We’ve since automated the path from specification to ship, which only moved the constraint. The human reviewers are now the bottleneck.

So, we’re teaching agents to out-review humans. Different people ask different review questions, so we aggregate them into a checklist the agents run every time. Then we ask Claude for other industry review best practices and teach those too. Eventually, an agent reviews things 100 ways automatically, leaving less that requires a human eye. A well-trained agent carrying the load, with humans doing the high-value assessment, beats what a human could do alone. No one has time for those 100 checks plus their own advanced 10.

A human still signs off on everything

For oversight, we use a Kanban board the team has run for years to track progress. Treat agents like team members and the rest follows: prioritize their work, scope it clearly, check progress, review output and iterate. Code from one agent gets reviewed by another agent running in a separate context window on a different underlying model. A pull request triggers another round, and the agents can spend an hour iterating until it’s clean. No agent opens a pull request without a human signing off first. The review is layered: subagents check Muffin’s work, Travis’s team checks engineering and the pull request gets a final pass. A detailed set of control policies, enforced by the agent harness, holds the security line — all written down and built into the agents’ instructions, so they know when and how to ask for human review.

One designer now does the work of three

Going from idea to prototype used to take a few weeks. Now, it takes about one day. We built new in-product dashboards measuring the value of AI work in 25 hours, back before our review processes were in place. Release velocity has doubled. We measure output weekly, with a metric that fits each function. For product management, it’s requirements defined; for UX, it’s interfaces designed. At the team level, we track the total capabilities we ship per period across every function. By that measure, each UX designer now produces what three used to. We also respond faster to feature requests, information requests and asks from sales enablement and marketing. 

This is also cheaper than what other companies are reporting. Most token-cost benchmarks measure a single developer writing code. My team spends $2,000 to $5,000 per person per month, covering a full role agent, plus specialized subagents for product, design and marketing — not just engineering. Measured against what that replaces, and against per-developer AI coding costs that Gartner says are on track to rival monthly salaries — with reports of single developers consuming $20,000 in a month — it’s reasonable. We’re deliberate about cost, routing simple prompts to cheaper models, and running small models on laptops where tokens cost us nothing. Together, these gains have let us defer hiring new staff.

For leaders who want to try this, here are three pieces of advice:

1. Study what peers are doing in any organization. There are AI black belts publishing best practices. Learn from them, then lead your team.
2. Push your teams to collaborate early. It keeps work aligned with production standards, so projects don’t have to be rebuilt, and produces higher quality faster.
3. Don’t start from scratch. The space moves fast enough that plenty of tools already exist to build on. You’ll still need to shape them to how your team works, but you don’t have to be an expert to start.

The bigger change has been cultural. People stay experts in their own lanes, but that expertise now guides and refines work others used to own outright. As Katie puts it: “The net effect is we all end up acting more like product owners, not just holders of one job title.”

In the end, everyone becomes a manager of agents — in any role.

Every AI roadmap review I sit through this year eventually lands on the same question: which technology stack/model should we use? It’s the wrong first question to start with. Gartner projects that by 2027, organizations will deploy small, task-specific AI models at roughly three times the volume of general-purpose large language models. In my experience running agentic pilots across a large enterprise, those specialized models succeed or fail almost entirely on something that has nothing to do with which model you picked: whether your own business actually agrees on what its data means.

That’s a harder problem than it sounds, and it’s not the same problem as data quality. My team has spent years measuring data quality the traditional way — is the field populated, is the number accurate, is the record deduplicated. Those checks matter, but they answer a narrower question than the one agentic AI actually depends on. IBM’s own explainer on the topic puts it well: a semantic layer is the piece of enterprise architecture that translates complex, technical data structures into the business terms people actually use day to day. Clean data can still be semantically ambiguous. A field can be perfectly accurate and still mean three different things depending on which team is looking at it.

Why accurate and understood are not the same thing

I recently worked with a large MedTech firm where I led data and governance strategy to build AI agents; the term “active customer” existed in three separate systems, defined three separate ways: one team counted anyone with an account open in the last 90 days, another counted anyone with a transaction in that window, and a third counted anyone who hadn’t formally churned. Every one of those definitions was technically accurate — the underlying data was clean, well-governed and correctly populated. None of them agreed with each other.

For years, that ambiguity lived quietly inside dashboards, where a human analyst could squint at a number, ask a colleague which definition it used and adjust their interpretation accordingly. Agentic AI removes that human buffer. When an agent pulls “active customer” to decide who gets a retention offer or who gets flagged as a churn risk, it doesn’t know there are three competing definitions sitting behind that phrase — it picks whichever one the underlying table happens to expose and acts on it with full confidence. The output looks decisive. It’s just decisively wrong for two-thirds of the organization.

What happens when agents inherit your organization’s ambiguity

I learned this the expensive way. When we ran an early agentic pilot on a customer retention workflow last year, the agent flagged a segment of accounts as “at-risk” that the finance team didn’t recognize as at-risk at all. It wasn’t a model problem — the reasoning was sound given the data it was pointed at. It was a “definitions” problem: sales, finance and customer support each had their own notion of “at-risk,” built into three different systems at three different points in the company’s history, and nobody had ever forced them to reconcile. We ended up spending more time aligning those definitions across departments than we spent building the agent itself.

That experience isn’t an outlier. Informatica’s 2026 survey of 600 global data leaders found that half cite data quality as their top barrier to deploying agentic AI, and 57% call data reliability the wall standing between a pilot and real production use. In my own postmortems, a meaningful share of what gets logged as a “data quality” issue in surveys like that is really a semantic issue wearing a data-quality costume: the data was accurate; it just didn’t mean what the agent — or the team reviewing its output — assumed it meant.

This matters more for agentic systems than it ever did for reporting, because agents don’t just surface an ambiguous number for a human to sanity-check. They act on it. A dashboard with a wrong “at-risk” count is an annoyance. An agent that auto-triggers retention offers, or worse, credit decisions, based on the wrong definition of “at-risk” is a business risk with a dollar figure attached.

The same pattern shows up anywhere a term carries more weight than its plain-English simplicity suggests. “Revenue” means something different to a sales team tracking bookings than it does to finance tracking recognized revenue under accrual accounting — a distinction that rarely causes trouble in a quarterly business review, where a human in the room can ask which number is being quoted. Hand that same ambiguity to a pricing or forecasting agent, and it will pick one definition, apply it consistently and confidently, and generate a forecast that’s internally coherent and organizationally wrong. The failure mode isn’t a crash or an error message. It’s a plausible-looking number that nobody thinks to question until it’s already driven a decision.

Build the semantic layer before you scale the next model

The fix isn’t a bigger model, and it isn’t even primarily a bigger data quality initiative, though that’s the instinct most organizations reach for first. It’s a deliberate semantic layer — the shared, machine-readable set of business definitions that sits between your raw, clean data and whatever model or agent is querying it. In practice, that means going through a sequence most data teams skip because it’s slower and less exciting than deploying the next pilot.

Start by auditing and cataloging the terms most central to your highest-priority use case — not every term in the business, just the three or four that the agent will actually touch. Force the departments that use those terms to sit in a room and agree on one definition, even when that conversation is uncomfortable and political, because it usually is. Then encode that agreed definition somewhere machine-readable and enforced — not just written down in a glossary document nobody opens again after the kickoff meeting — so every model, dashboard and agent that touches that term resolves it the same way.

None of this needs to wait for a multi-year data governance overhaul, and I’d actively discourage treating it that way. The mistake I made early on was assuming semantic work had to be comprehensive to be useful — mapping every term across every system before touching a single agent. It doesn’t. Scoping it tightly to one workflow, getting the two or three business functions that actually touch that workflow to agree, and encoding that narrow agreement is a project measured in weeks, not quarters. The comprehensive enterprise glossary can, and probably should, come later, built one validated use case at a time rather than designed top-down in a conference room before anyone has tested it against a real agent.

CDOs and CIOs would have to ensure that this is also a mandatory shift for data teams themselves, not just a technical build. The job used to be building reports and charts for humans to interpret with their own judgment layered on top. It’s increasingly about building and maintaining the context that lets a machine interpret data correctly without a human in the loop to catch the ambiguity. That’s a different skill set than most data teams were hired for, and it’s worth saying so explicitly when planning headcount and training for the next year, rather than assuming the existing analytics team will absorb it by osmosis.

The organizations that come out ahead in the next phase of agentic AI won’t necessarily have access to a better model than their competitors — most of us will be drawing from a similar, increasingly commoditized set of models within the next year or two. What they’ll have is a semantic foundation clear enough that a smaller, specialized model can act on it with confidence, and a team that already went through the unglamorous work of agreeing, in writing, on what the business actually means by its own words. That’s a much harder thing for a competitor to copy than a model subscription — and it’s the argument I now make first in every AI budget conversation, well before anyone asks me which model we’re using.