By Marshall Heilman, CEO, DTEX
I expect many 2027 security predictions to focus on how attackers are using AI. They are, and those attacks will always capture headlines. But the incident I expect to do the most damage will be one the company set up itself, by putting an agent into production and trusting that no harm would be done in the completion of the task.
The warning signs are already public. OpenAI, Anthropic and outside researchers have reportedly investigated tens of thousands of cases involving concerning model or agent behavior. The figure includes activity observed in testing as well as real-world use, and not every case resulted in harm. Even so, the scale shows that unexpected agent behavior, or agent behavior with unintended consequences, is no longer an edge case.
Enterprises have spent years learning how to investigate human behavior. They have far less experience tracing how an agent interpreted a request and why it chose particular actions to complete that request. We are all especially inexperienced at investigating and understanding why agents will work together to circumvent control to accomplish an object, even if outside the scope of the intended outcome. That gap in experience will define many of the AI security challenges enterprises face in 2027.
1. The first major AI-caused business outage will come from an agent following authorized instructions
I predict 2027 to bring the first widely reported incident in which a company-deployed agent disrupts a critical business operation. No attacker will be involved. An agent will pursue a legitimate goal and take actions nobody would have approved.
Picture an agent told to disable accounts inactive for 90 days. It does exactly that. Among them are the service accounts that run payroll, the integration account behind single sign-on, and the mailbox that receives customer orders. None of them “log in” the way the instruction meant. The agent finishes, posts a summary of 412 accounts disabled, and the company spends the next day finding out why nothing works. The example is hypothetical, but the pattern is not: Replit’s agent deleted a production database during a code freeze, while OpenAI agents accessed an outside company while pursuing evaluation goals in ways their operators did not intend.
Unlike traditional automation, which follow a sequence of steps, an agent may interpret a goal and decide how to complete it in ways the human authorizing the action did not intend or even consider.
2. The human-in-the-loop will become the weakest link
I expect attackers to deliberately target the people who approve agent actions. Some approvers will develop approval fatigue and start rubber-stamping requests without much thought. Attackers will also find ways to approve actions on the authorizing person’s behalf, and social engineering will rise sharply but with significantly higher stakes.
Human approval is one of the most common safeguards for agents running in production business environments. But as with any other technology system, people will be the weakest link. Picture an agent asking for approval on routine tasks 20 or more times a day. The approver is unlikely to scrutinize every request, every day. The same problem shows up when agents ask for approval at odd hours or send requests to backup authorizers. Human-in-the-loop could become another check-the-box safeguard that doesn’t deliver the protection people expect.
Attackers will adapt and take the easiest path to their objective. It’s usually easier to trick a person into approving a request than to manipulate an agent into taking an unauthorized action.
Organizations need to build behavioral analysis into how they govern and oversee agents. That way, people are only asked to approve activity that falls outside expected behavior. I expect organizations to adopt tiered risk models, where only the highest-risk requests go to a human for review.
3. Permissions alone won’t keep agents in check
Gartner predicts that by 2027, 40% of enterprises will demote or decommission autonomous AI agents after production incidents expose governance gaps. I think that’s a reasonable call, and permissions are a big part of why.
Many agents inherit the permissions of whoever deployed them, which often leaves them over-permissioned from day one. Tighter scoping helps, but once an organization runs hundreds or thousands of agents, getting permissions exactly right for every scenario isn’t realistic. This simply magnifies the over-permissioning problem organizations already face. Just-in-time access doesn’t close the gap either, because an attacker hijacking an agent only needs it to do the wrong thing at that exact moment.
Security spending will shift toward watching what agents do while they run, and blocking actions that don’t fit the request. The focus will be on the intent of the authorized action and whether the agent is acting within the bounds of that intent.
4. Agent accountability will reach the Board room as a matter of risk control
When Microsoft’s CEO calls AI an insider risk, the debate over whether agents are insiders is largely settled, and as DTEX CTO Rajan Koo explains, the playbook to manage that risk already exists. The next question is who answers for them. In 2027, accountability will follow delegated authority for computer use agents, and multiple c-suite executives for production and business use agents.
I expect organizations to treat each employee and the agents they direct as a single unit. If you give an agent broad access to do a narrow job, you share responsibility for what it does, the same way you would if you handed someone your password. That doesn’t mean blaming people for every unexpected step. It means being able to show what they asked for, what authority they gave, and how closely they paid attention to the agent’s actions. Organizations will need to trace an agent’s actions back to the person who set it in motion and the authority that made those actions possible.
It also covers deliberate misuse. In one DTEX investigation, a system administrator who wanted to steal data created an agent under a colleague’s account, so the activity would point at someone else. He told it to collect and obfuscate the data. The agent had no encryption tool available, so it downloaded code from the internet to build one. That code had multiple vulnerabilities. The agent completed its job, implicated an innocent employee, and left the company less secure.
For agents deployed by IT in production or to assist with internal business operations, I expect a c-suite executive to share final accountability for them with the CISO, most likely the CIO. This is a departure for how cyber security accountability is handled with human behavior, where the CISO often has sole final accountability when a cyber security incident occurs. This shared responsibility makes sense because the CISOs job is to manage the risk from the agent’s actions, and the CIO’s job is to ensure agents are deployed, configured, and operating properly.
5. Nation-states will start using your agents against you
AI-enabled nation-state operations are already here. Anthropic’s September 2026 threat intelligence report describes a Russian espionage actor whose AI agents rebuilt and redeployed its malware whenever security products detected it. I expect more state actors using internally deployed AI capabilities against a victim organization to be publicly reported in 2027.
The bigger shift comes after they get in. A compromised employee account may provide access to every agent that employee can invoke. I expect attackers to use victims’ internal agents to find and steal what they want. Every security team should know today which agents a compromised account could command.
North Korea is where I’d watch first. Its operators have a long record of stealing directly from financial institutions after getting inside their networks, and they’ve since spent years placing IT workers inside companies. A July 2026 multinational alert warns that those workers use AI to disguise their identities and pose an insider threat involving data and cryptocurrency theft.
I also predict that we’ll see an attacker leverage an organization’s internal AI capabilities to learn how the organization operates (or how a complex system or process operates) and then to take malicious actions on behalf of the attacker.
Who owns this
AI agents are delivering real value, and companies will keep putting more of them into production in 2027. They should. But agents act at machine speed and without much oversight and ability to course correct. Organizations that can only reconstruct their actions after an incident will incur the most damage from unintended consequences and malicious actions.
The risk from agents also doesn’t belong only to the CISO or the CIO. It sits with the CEO and the board. No company would give a new employee broad access without oversight. Agents should not receive a lower standard, especially when they can move faster than the people supervising them.
