Visualização de leitura

Guild Group’s Mohammad Arif on the Security Risks of Enterprise AI

The Cyber Express Mohammad Arif

Every enterprise wants the upside of AI — faster workflows, sharper decisions, leaner teams. Far fewer have asked the harder question: what happens when the same systems delivering that upside are also quietly rewriting who, or what, has access to the crown jewels? Autonomous agents are now reading contracts, touching customer data, writing production code, and triggering workflows once reserved for trusted employees. The attack surface hasn't just grown — it's changed shape entirely. Mohammad Arif, Head of Information Security at Guild Group, has been on the front line of that shift, helping enterprise leaders separate genuine AI-driven cyber risk from the noise around it. In this interview with The Cyber Express, he makes the case that AI security isn't a niche technical concern to be handed off to a working group — it's a boardroom issue, sitting at the intersection of data protection, vendor risk, identity, and human accountability. His warning is blunt: organisations still treating AI security as tomorrow's problem are already behind, and the cost of catching up only rises from here.

The Cyber Expres: Everyone is talking about AI productivity. From a security leadership perspective, what is the biggest shift AI is creating for enterprises?

From a security leadership perspective, the biggest shift is that AI is changing the enterprise trust model. It is no longer only about protecting systems, networks, and users. Organisations now need to consider what AI can access, what data it can process, what decisions it may influence, and what actions it may trigger. AI can significantly improve productivity, cyber defence, and business decision-making. But the risk increases when adoption moves faster than governance. If AI tools are connected to sensitive data, enterprise workflows, identity systems, APIs, or business processes without clear controls, the risk is no longer just a technology issue. It becomes a data, privacy, operational resilience, and trust issue. So the real shift is this: AI is not just another productivity tool. It is becoming part of the enterprise operating model, and therefore it needs to be governed and secured like any other critical capability.

The Cyber Express: What worries you most about the current pace of AI adoption across organisations?

What worries me most is the gap between the speed of AI adoption and the maturity of AI governance. Many organisations are moving quickly to unlock productivity benefits, but they may not yet have clear rules around approved use cases, sensitive data handling, vendor assurance, retention of information, model outputs, and human accountability. The immediate risk is uncontrolled or "shadow" AI adoption, where employees or teams use public or unapproved AI tools without understanding what data is being entered, how that data may be retained, or whether it could be used to improve external models. This can create confidentiality, privacy, intellectual property, and regulatory risks. The challenge for enterprises is not to slow down innovation unnecessarily. The challenge is to enable AI safely — giving employees approved pathways to use AI, while putting the right controls around data protection, access, monitoring, and risk-based governance.

The Cyber Express: At what point does AI stop being just a productivity tool and become a real enterprise security challenge?

AI becomes a real enterprise security challenge when it moves from simply generating content to being connected to enterprise data, identity, applications, APIs, tools, and workflows. A standalone AI assistant used for drafting general content has a different risk profile from an AI agent that can access customer information, analyse internal documents, write code, trigger workflows, or make recommendations that people rely on. The risk increases further when AI has autonomy, can call external tools, or can operate across multiple systems. This is where agentic AI becomes important. The question is no longer only "what can the model say?" It becomes "what can the AI access, what can it do, and who is accountable for the outcome?" That is the point where traditional security controls need to extend into AI governance, identity, permissions, logging, monitoring, and human oversight.

The Cyber Express: Do you think most organisations are prepared for AI-driven cyber threats, or are many still treating AI security as a future problem?

Preparedness varies significantly. Some organisations are taking AI security seriously and are building governance, security review, data protection, and monitoring into their AI adoption programs. But many are still treating AI security as a future problem, even though AI is already inside the enterprise through productivity tools, SaaS platforms, coding assistants, analytics tools, third-party services, and employee-led experimentation. The issue is that AI adoption is often decentralised. It may start in business teams, technology teams, vendors, or individual users before the organisation has a complete view of the risks. That means security leaders need to shift from reactive control to proactive enablement. AI security readiness should include clear acceptable-use guidance, approved tools, data classification, vendor due diligence, secure development practices, incident response scenarios, awareness training, and monitoring. Without those foundations, organisations may not know where AI is being used, what data is exposed, or where the risk is accumulating.

The Cyber Express: What AI-related security incidents do you realistically expect enterprises to face over the next 12 months?

Over the next 12 months, I expect most enterprises to see AI-related incidents in a few practical areas. The first is AI-assisted social engineering. Phishing, impersonation, business email compromise, and executive fraud will become more convincing because AI allows attackers to generate personalised and credible content at scale. The second is sensitive data leakage into AI tools, which may happen when employees enter confidential business information, customer data, source code, contracts, security information, or internal documents into tools that have not been approved or assessed. The third is insecure AI integration. As teams connect AI to internal knowledge bases, applications, plugins, and workflows, weaknesses such as prompt injection, excessive permissions, poor output validation, or weak monitoring may create new attack paths. The fourth is AI supply-chain risk. Organisations increasingly rely on third-party models, APIs, datasets, plugins, open-source components, and AI-enabled SaaS platforms — each introducing dependencies that need to be assessed and monitored. So the threat is not one single scenario. It is an expanded attack surface across people, data, applications, vendors, and business processes.

The Cyber Express: AI-generated phishing and deepfakes are receiving significant attention. Have these threats become genuinely dangerous, or are they still more hype than reality?

They are genuinely dangerous, but the risk needs to be understood properly. The concern is not only that AI can create fake emails, voices, images, or videos. The bigger issue is that AI reduces the effort required to create believable, personalised, and scalable deception. Traditional phishing often had indicators such as poor grammar, generic wording, or obvious formatting issues. AI reduces those indicators. Attackers can tailor messages to specific roles, business processes, recent events, or organisational language. Deepfakes also create risk where organisations rely heavily on voice, video, or informal executive instructions for approvals or sensitive actions. This does not mean every organisation will face sophisticated deepfake attacks immediately. But it does mean that trust-based processes need to be strengthened. Payment approvals, changes to bank details, privileged access requests, sensitive data transfers, and executive instructions should have strong verification controls that do not rely on one communication channel alone.

The Cyber Express: Do enterprises fully understand the risks of feeding sensitive data, business context, or internal information into frontier AI systems?

Not always. Many organisations understand the general concern, but they may not fully understand the practical ways sensitive information can be exposed through AI usage. The risk is broader than simply entering customer data into a public tool. Employees may enter internal strategies, contracts, security designs, source code, incident information, board papers, commercial terms, or confidential business context. Even if the data is not used for model training, there may still be risks around retention, access, logging, jurisdiction, vendor terms, and downstream use. Enterprises need to apply the same discipline to AI that they apply to other sensitive platforms — understanding what data is allowed, which tools are approved, whether enterprise-grade privacy and security settings are enabled, how data is retained, and whether the vendor's contractual terms align with the organisation's obligations. The practical starting point is data classification. If organisations do not know what data is sensitive, they cannot consistently govern how that data should or should not be used with AI.

The Cyber Express: We have seen supply-chain attacks target software and open-source ecosystems. Could AI models, tools, plugins, agents, datasets, and integrations become the next major supply-chain risk?

Yes, AI supply-chain risk is likely to become a major area of focus. In traditional technology environments, organisations already assess software vendors, cloud providers, managed service providers, open-source libraries, and third-party integrations. AI expands that supply chain. The AI supply chain can include foundation models, fine-tuned models, datasets, model providers, APIs, plugins, orchestration platforms, vector databases, prompt libraries, AI coding tools, agent frameworks, and embedded AI features in SaaS products. A weakness or compromise in any of these areas can affect the confidentiality, integrity, or reliability of AI-enabled systems. A poorly governed dataset could introduce bias or inaccurate outputs. A vulnerable plugin could expose data. A compromised package could affect an AI-enabled application. An over-permissioned AI agent could access more information than it needs. These are not only technical risks; they are third-party, operational, and governance risks. Enterprises should therefore treat AI supply-chain assurance as part of broader cyber risk and vendor risk management, including due diligence, contractual controls, security testing, data protection review, monitoring, and clear accountability between the organisation and its providers.

The Cyber Express: How should security leaders rethink traditional cybersecurity strategies in a world where AI can write code, analyse vulnerabilities, automate tasks, and support attackers as well as defenders?

Security leaders should not abandon traditional cybersecurity principles. Instead, they need to extend them into AI-enabled environments. Identity and access management remains critical, but now we need to consider identities and permissions associated with AI agents, service accounts, APIs, and automated workflows. Data protection remains critical, but now we need to monitor prompts, outputs, embeddings, and knowledge retrieval systems. Secure development remains critical, but now we need to assess AI-generated code, model behaviour, prompt injection risks, and third-party AI components. The same applies to monitoring and incident response. Security teams need visibility into AI usage, unusual activity, sensitive data exposure, misuse of AI tools, and unexpected agent behaviour. Incident response plans also need to consider AI-specific scenarios such as data leakage through AI platforms, compromise of AI integrations, prompt injection, poisoned data, or misuse of AI-generated code. The key point is that AI security should not sit outside the cybersecurity operating model. It should be integrated into governance, architecture, procurement, engineering, monitoring, awareness, and incident response.

The Cyber Express: Are existing cybersecurity teams equipped to handle AI-era threats, or does the industry need new skills and operating models?

Existing cybersecurity skills remain highly relevant, but they need to be expanded. The fundamentals still matter: identity, data protection, secure architecture, vulnerability management, incident response, third-party risk, and governance. However, AI introduces new concepts that security teams need to understand. Security professionals now need AI literacy — how large language models work at a practical level, how AI applications are integrated, how prompts and outputs can be manipulated, how retrieval-augmented generation works, how AI agents interact with tools, and how AI supply chains are structured. The operating model also needs to evolve. AI security cannot be owned by security alone. It requires collaboration between cybersecurity, technology, data, privacy, legal, risk, procurement, HR, and business teams. In many organisations, the most effective model will be a cross-functional AI governance group supported by security-by-design processes. So yes, the industry needs new skills, but not at the expense of existing cybersecurity disciplines. The future is a combination of traditional cyber expertise, AI literacy, risk management, and business enablement.

The Cyber Express: AI vendors promise speed, scale, and automation. Where should organisations avoid over-relying on AI in cybersecurity and business decision-making?

Organisations should avoid over-relying on AI in areas where decisions are high-impact, sensitive, difficult to reverse, or require strong judgement. AI can assist, but accountability should remain human. In cybersecurity, this includes incident severity decisions, containment actions, identity and privileged access approvals, regulatory interpretation, legal assessments, and decisions that could materially affect customers, employees, or critical operations. AI can help summarise information, detect patterns, prioritise alerts, and recommend actions, but those recommendations should be validated by qualified people. There is also a risk of automation bias, where people trust AI outputs because they appear confident or well-structured. That can be dangerous if the output is incomplete, inaccurate, or based on weak context. Organisations need clear rules for where AI can automate, where it can recommend, and where human approval is mandatory. A practical principle is this: the greater the potential impact, the stronger the need for human oversight, auditability, and accountability.

The Cyber Express: If you were advising enterprise leaders today, what are the first three things they should do to prepare for the security impact of frontier AI models?

The first priority is to establish AI governance. Organisations need clear policies on approved use cases, acceptable tools, data handling, human oversight, and accountability. Governance should not be theoretical — it needs to be embedded into procurement, technology delivery, data management, security review, and business processes. The second priority is to protect sensitive data before scaling AI adoption. This means strengthening data classification, access controls, data-loss prevention, retention rules, and monitoring. Enterprises should know what data can be used with AI, under what conditions, and through which approved platforms. The third priority is to integrate AI into the cyber risk management operating model — including third-party risk assessments, threat modelling, secure development, incident response, employee awareness, logging, monitoring, and assurance activities. Security teams should also start preparing for AI-specific risks such as prompt injection, insecure integrations, sensitive data exposure, excessive agency, and AI supply-chain compromise. AI can create significant value, but only if organisations adopt it with discipline. The organisations that treat AI security as a future issue may already be behind.
"AI adoption without governance is not innovation — it is unmanaged risk. The next phase of enterprise security is about controlling what AI can access, what it can do, and who remains accountable." — Mohammad Arif, Head of Information Security, Guild Group

NCSC Calls for Tight Security and Human Oversight as Agentic AI Use Expands

Agentic AI Deployment

The UK’s National Cyber Security Centre (NCSC) has warned organizations to take a measured approach toward adopting agentic AI, highlighting the growing cyber and operational risks associated with highly autonomous AI systems. In a new guidance document co-authored with international partners, the NCSC said businesses should avoid rushing into large-scale deployments of agentic AI tools without understanding the security implications. The guidance recommends starting with low-risk use cases, limiting system privileges, and maintaining strong human oversight throughout deployment. The advisory comes as organizations increasingly experiment with AI systems capable of making decisions, accessing tools, and carrying out actions with limited human involvement.

What Is Agentic AI?

Unlike traditional generative AI systems that primarily create text, images, or predictions, agentic AI systems are designed to independently pursue goals. These systems can access data sources, remember context, make decisions, interact with software tools, and even create sub-agents to complete tasks. According to the NCSC, this added autonomy is what makes agentic AI useful for areas such as cyber defense, workflow automation, and operational efficiency. However, it also introduces a wider attack surface and increases the difficulty of monitoring system behavior. The agency noted that many security risks linked to AI are not entirely new. Concerns around access control, supply chain security, monitoring, and incident response already exist in traditional IT systems. Agentic AI systems also inherit existing large language model risks, including prompt injection and jailbreaking attacks. However, the NCSC warned that the autonomy of agentic AI systems could amplify these issues, especially if organizations deploy them without proper safeguards.

Why Agentic AI Raises Security Risks

The guidance outlines several risks tied to agentic AI deployments. One of the main concerns is broader access to systems and sensitive data. AI agents may interact with external tools, APIs, or databases in ways that traditional AI applications do not. The NCSC also highlighted the possibility of unpredictable behavior. Since AI agents interpret goals autonomously, they may take actions that differ from human expectations or exceed their intended scope. Another challenge involves visibility and oversight. Autonomous systems can operate at speeds that make meaningful human review difficult, particularly in enterprise environments where multiple systems and workflows are interconnected. The guidance further noted that explaining the behavior of agentic AI systems can be more difficult than understanding conventional AI models. The combination of decision-making, tool usage, and autonomous actions creates additional complexity during incident investigations or compliance reviews.

NCSC Calls for Incremental Agentic AI Deployment

To reduce risks, the NCSC urged organizations to adopt agentic AI gradually instead of deploying it across critical systems from the outset. The guidance recommends tightly controlled pilot deployments focused on clearly defined, low-risk tasks. Organizations are also encouraged to assess whether AI is genuinely necessary before integrating autonomous agents into existing workflows. “If you cannot understand, monitor or contain an agent’s actions, it is not ready for deployment,” the guidance stated. The agency stressed that organizations should never grant unrestricted access to sensitive data or critical infrastructure. Maintaining visibility into AI system behavior and preserving meaningful human control were identified as key requirements for safe deployment.

Human Accountability Remains Essential

Despite the growing capabilities of autonomous AI systems, the NCSC emphasized that humans remain fully accountable for how these technologies are used. The guidance states that organizations should clearly define who is responsible for approving AI access, monitoring system behavior, reviewing incidents, and shutting systems down when necessary. Security teams were also advised to integrate agentic AI risk management into existing cybersecurity and governance frameworks instead of treating AI security as a separate process. Recommended practices include applying least-privilege access controls, limiting system scope, avoiding long-lived credentials, monitoring unusual behavior, and planning for incidents involving AI misuse or loss of control.

Path Forward

While warning about the risks, the NCSC acknowledged that agentic AI could deliver significant operational benefits, particularly for repetitive and low-risk tasks. The agency said organizations should focus on responsible and scalable adoption strategies built around existing cybersecurity practices and strong governance controls. The guidance ultimately encourages businesses to move carefully, test systems incrementally, and prepare for potential failures before expanding the role of autonomous AI systems across enterprise environments.
❌