As businesses rush to deploy autonomous AI agents across operations, a growing gap is emerging between innovation and oversight. A new report from Deloitte warns that organisations are embracing agentic AI far faster than they are building the governance, security, and accountability frameworks required to control it.
The result, Deloitte argues, is a widening risk landscape where AI agents increasingly operate inside production systems without the safeguards needed to ensure trust, compliance, and resilience. While the technology promises major productivity gains, insufficient governance could expose businesses to data breaches, operational failures, regulatory penalties, and reputational damage.
The message from Deloitte is clear: speed alone is not a strategy.
AI Agents Move From Pilots to Production—Too Fast
According to Deloitte’s findings, agentic AI systems are transitioning from experimentation to real-world deployment at unprecedented speed. Unlike earlier waves of automation, AI agents are now making decisions, triggering workflows, and interacting with sensitive systems with minimal human involvement.
However, the risk frameworks used by many organisations were designed for human-led processes, not autonomous digital actors.
Traditional controls—such as manual approvals, static access permissions, and post-incident audits—are struggling to keep pace with agents that can operate continuously, scale instantly, and adapt dynamically.
Deloitte’s survey data highlights this imbalance:
- Only 21% of organisations report having strong governance or oversight mechanisms in place for AI agents
- 23% of businesses say they are already using AI agents today
- That figure is expected to surge to 74% within the next two years
- Meanwhile, the proportion of companies with no AI agent adoption is projected to fall from 25% to just 5%
In short, adoption is accelerating far faster than safety.
The Real Risk Is Not AI—It’s Weak Governance
Deloitte is careful to stress that AI agents themselves are not inherently dangerous. Instead, the threat lies in how they are deployed.
When agents are allowed to operate with vague objectives, excessive permissions, or little visibility, their behaviour can quickly become opaque. In these conditions, organisations struggle to understand why decisions were made, who—or what—was responsible, and how to prevent similar issues in the future.
Without governance, accountability breaks down. And when accountability breaks down, risk becomes difficult to manage and nearly impossible to insure.
Governed Autonomy, Not Unchecked Automation
Ali Sarrafi, CEO and founder of enterprise AI firm Kovant, argues that the solution is not to slow innovation—but to constrain it intelligently.
“The goal should be governed autonomy,” Sarrafi explains. “AI agents need clear boundaries, defined policies, and escalation paths—just like human employees.”
In this model, agents are permitted to move quickly on low-risk tasks within strict guardrails. When actions approach higher risk thresholds, control shifts back to humans.
“Well-designed agents don’t replace oversight,” Sarrafi says. “They work within it.”
Why Real-World Environments Break AI Agents
AI agents often perform impressively in demos and controlled environments. But business systems are rarely clean, consistent, or predictable.
Enterprise environments typically involve:
- Fragmented systems
- Inconsistent data quality
- Legacy software
- Conflicting permissions
- Complex compliance requirements
In these conditions, AI agents can behave unpredictably—especially if given too much context or authority at once.
“When agents are exposed to overly broad scope, they become prone to hallucinations and unexpected actions,” Sarrafi notes. “That’s when things go wrong.”
Designing for Predictability, Not Just Capability
Production-grade AI systems take a different approach. Rather than deploying a single, all-powerful agent, they break work into narrow, well-defined tasks handled by specialised agents.
This structure offers several advantages:
- More predictable behaviour
- Easier monitoring and debugging
- Clear traceability of decisions
- Faster detection of failures
When something does go wrong, issues can be isolated and addressed before cascading across systems.
“This kind of architecture makes intervention possible,” Sarrafi explains. “Instead of reacting to disasters, teams can catch problems early.”
Accountability Makes AI Insurable
As AI agents begin to take real actions—updating records, approving transactions, triggering workflows—questions of liability become unavoidable.
For insurers, opaque AI systems represent unacceptable risk.
That changes when agents operate with:
- Detailed action logs
- Clear decision pathways
- Human approval for high-impact actions
- Replayable workflows
When every action is recorded and attributable, risk becomes measurable.
“This transparency is essential,” Sarrafi says. “It allows insurers, auditors, and regulators to understand what happened and why.”
In other words, accountability turns AI from a black box into something that can be evaluated, insured, and trusted.
Standards Help—but They Are Not Enough
Shared technical standards are beginning to emerge for agentic AI, including those being developed by the Agentic AI Foundation (AAIF). These initiatives aim to improve interoperability between different agent systems.
However, Sarrafi cautions that many standards focus on what is easiest to build—not what enterprises need to operate safely at scale.
“Enterprises don’t just need agents to talk to each other,” he says. “They need standards that support operational control.”
That means frameworks that include:
- Fine-grained access permissions
- Approval workflows for sensitive actions
- Comprehensive logging
- Built-in observability
Without these features, interoperability alone does little to reduce risk.
Identity and Permissions: The First Line of Defence
One of the most critical controls in AI agent deployment is identity management.
Every agent should have:
- A clear identity
- Explicit permissions
- Well-defined action boundaries
“When agents are granted broad privileges or excessive context, they become unpredictable,” Sarrafi warns. “That’s when security and compliance risks emerge.”
Limiting what agents can see and do is not a constraint—it’s a safety feature.
Visibility Builds Trust Across the Organisation
Strong governance is impossible without visibility.
When every agent action is logged and observable, teams gain:
- Insight into system behaviour
- The ability to investigate incidents
- Confidence that controls are working
This transparency benefits not just IT teams, but also compliance officers, risk managers, executives, and insurers.
“Visibility transforms AI agents from mysterious components into systems you can inspect and audit,” Sarrafi explains. “That’s what builds trust.”
Deloitte’s Blueprint for Safe AI Agent Deployment
Deloitte’s report outlines a structured approach to AI agent governance designed to balance innovation with control.
At the core is the concept of tiered autonomy.
In early stages, agents may:
- Access information
- Analyse data
- Provide recommendations
As confidence grows, agents can be allowed to:
- Take limited actions with human approval
- Operate autonomously in low-risk areas
Only after demonstrating reliability should agents be trusted with higher-impact decisions.
This gradual approach allows organisations to scale safely while continuously validating performance.
Embedding Governance Into Daily Operations
Deloitte’s Cyber AI Blueprints recommend embedding governance directly into organisational controls rather than treating it as an external layer.
This includes:
- Built-in policy enforcement
- Continuous risk assessment
- Integrated compliance monitoring
- Clear escalation paths
By making governance part of everyday operations, organisations can respond to issues in real time rather than after damage has occurred.
Training People Is as Important as Training Models
Technology alone cannot ensure safe AI adoption. Deloitte emphasises the importance of workforce readiness.
Employees need to understand:
- What data should never be shared with AI systems
- How to recognise abnormal agent behaviour
- What steps to take when systems behave unexpectedly
Without proper training, even well-designed controls can be undermined by human error.
“If people don’t understand the risks, they may unintentionally bypass safeguards,” Deloitte warns.
From Innovation Race to Trust Race
As AI agents become more capable, competitive advantage will no longer be defined by who deploys them first—but by who deploys them responsibly.
Deloitte’s message is that visibility, governance, and accountability will separate leaders from laggards.
“Companies that prioritise control and trust will outperform those that prioritise speed alone,” the report suggests.
A Critical Moment for Enterprise AI
AI agents are poised to reshape how work gets done. But without robust guardrails, they also have the potential to magnify risk at machine speed.
Deloitte’s warning arrives at a pivotal moment. With adoption accelerating rapidly, organisations still have time to embed governance before AI agents become deeply entrenched in critical systems.
Those that act now—by investing in visibility, permissions, standards, and human oversight—will be best positioned to harness AI’s benefits without losing control.
In the age of autonomous systems, trust is not optional—it is infrastructure.
(Image source: “Global Hawk, NASA’s New Remote-Controlled Plane” by NASA Goddard Photo and Video is licensed under CC BY 2.0. )
Read Also: Gallup Workforce Survey Reveals How AI Is Really Being Used in US Workplaces