By Mal Vivek, CEO and Founder of zeb
AI agents are being deployed at breakneck speed across industries, but these systems are fallible. Such limitations leave leaders grappling with a question that most contracts and org charts were not built to answer. When an autonomous agent makes a catastrophic miscalculation, who is culpable?
As enterprises increasingly rely on autonomous AI agents, costly failures will raise complex questions about liability across model providers, software vendors and customers, making it essential for businesses to implement software-level guardrails and understand their responsibility before deploying AI at scale.
A costly mistake from an AI agent in 2027 is a matter of when, not if, and there are three potential targets sitting in the blast radius: the model provider, the software vendor and the customer. Each has a different level of exposure, but only one of them controls the layer where protection actually happens.
The agent will fail. The open question is who gets the blame.
The agent conversation has been dominated by capability. What can they do? How fast can they do it? What will it cost? Far less attention has been given to what happens when an agent gets it wrong.
In 2027 and beyond, that attention will become increasingly important as agents go beyond just generating text to actually taking actions. An agent that places an order, issues a refund or commits the organization to something is operating within your business, often without a human reviewing each step. The first major incident may be a quiet error repeated multiple times before anyone notices there is a problem, not a dramatic breach.
For the model provider, the software vendor and the customer, liability isn’t spread evenly. It will settle on who owns the product.
Model providers built the engine, not the car
Of the three, the model provider is the least likely to be in the line of fire because of structural reasons. Model providers offer general-purpose capabilities but don’t build the actual product that takes action within the business, set the permissions that allow it to act or pick the workflows it touches. The owners of that product are the ones who will have the most liability for what it does.
A major security incident would be the exception. If the failure originates in the provider’s systems, then the provider may be on the hook. Barring that, they are rarely the ones with the burden of responsibility.
But you shouldn’t treat the model’s maker as a scapegoat. If your plan for a bad outcome is that the provider will handle it, you don’t actually have a plan.
SaaS liability and accountability
Many enterprises will work with agents via software they bought rather than built. The AI is embedded in the SaaS product, and the customer doesn’t have control over what’s inside.
If that agent fails, both the buyer and the vendor will take a hit. The divide boils down to one question: How much of the failure can be traced back to your configuration, and how much can be traced back to the black box of the vendor’s software?
You can’t audit what you can’t see, so that should influence how you buy. The part you do control — permissions, workflows, settings — should be the part that gets examined first. Every setting you touch should be deliberate and explainable, treated as a decision you may one day have to defend.
A custom build is your product
If a partner builds a system for you, the work product transfers to the customer at creation. It is their IP and their product. With ownership comes stewardship.
But the builder isn’t completely off the hook. There is some professional-errors exposure, which is what a service provider’s errors and omissions insurance covers, but that has a limit. The primary implicated party will still be the customer because they didn’t have the proper AI guardrails and operational controls around what they now own.
Though it’s a different route than a SaaS purchase, it reveals the same pattern. Whether your agent was purchased or commissioned, liability will follow ownership, and if you’re not prepared for that as we enter 2027, you’re already behind.
Controlling the protection layer
The model provider sits far away from the action. The vendor only controls its own software. But the customer holds the permissions, workflows and authority the agent acts under.
Though the customer is the most exposed, the customer also has most leverage. Protection lives in the software layer above the model. This is where limits are set on what the agent can do without human sign-off as well as the thresholds that trigger human review and logging that catches errors early.
But remember that guardrails you can’t explain aren’t actually guardrails. In 2027, owning the layer means being able to explain how the model behaves inside of it. If you can’t, that’s the work. Black box operations don’t do anyone any good.
Capability has become something purchasable, but accountability has not. The company that owns the product owns what it does, so the key to preventing agentic vulnerabilities like expensive mistakes is to build the guardrails before you need them.
##
ABOUT THE AUTHOR
Mal Vivek is an entrepreneur, technologist, and business builder who has spent her career at the intersection of emerging technology and organizational transformation. As founder and CEO of zeb, she has built a reputation for challenging conventional thinking about how companies adopt and scale new technologies. Her work focuses on helping organizations navigate periods of technological disruption, translating complex advances in AI into practical business outcomes. Known for her systems-level thinking, Mal’s perspective centers on a simple belief: transformative technologies create value only when organizations are willing to rethink the way they operate.
Over the last several years, Mal has guided zeb through its evolution from a digital transformation consultancy into an AI-native organization built from the ground up for the age of artificial intelligence. Rather than layering AI onto existing processes, she led the company through a complete redesign of its structure, delivery model, and operating philosophy, culminating in the development of Substrate, zeb’s AI-native operating architecture. Today, she is a sought-after voice on AI transformation, the future of work, organizational design, and the changing economics of professional services, arguing that the biggest competitive advantage in the AI era won’t be access to better technology, but the willingness to build fundamentally different organizations around it.
