By Nitish Mane, Software Engineer at Philo
Sometime in 2027, an AI agent will be the first responder on a lot of pages. It will pick up an alert, pull the relevant metrics and logs, form a theory about what broke, and draft a fix before the engineer on call has finished reading the notification.
My prediction is narrower than “agents will run operations.” It is that bounded autonomy becomes standard platform practice. The teams that do well with agents on call will treat the agent’s autonomy as something to scope and limit, the same way they scope a service account or a deploy pipeline. The teams that treat autonomy as the feature to show off will learn the same lessons through incidents.
I build agents that diagnose production alerts and propose fixes behind human approval gates. Before that, I helped run large-scale cloud infrastructure, including a GitOps control plane that managed 34 Kubernetes clusters. What follows comes from both.
What an agent on call actually does
The most useful on-call agent shortens the time between an alert firing and a person making a good decision. Fixing things on its own is a secondary concern. In practice the loop has four parts.
1. Diagnose. The agent reads the alert, the recent changes and the telemetry around it, then writes down what it thinks is wrong and why. The reasoning matters as much as the conclusion, because the reviewer needs to check the logic as well as the answer.
2. Propose in a sandbox. The fix is drafted where it cannot hurt anything: a branch, a pull request, a dry run against a copy of the environment. The agent can test its idea there as often as it likes.
3. Wait at the gate. Nothing reaches production until a person approves it. The approval is a real review, with the diagnosis, the diff and the expected effect in front of the reviewer.
4. Know the way back. Every proposed change carries a rollback path that was worked out before the change shipped, not after it failed.
The agent will be wrong some of the time. It will match a familiar pattern to an unfamiliar problem, or fix the symptom and miss the cause. The design has to assume that. With the gate and the rollback in place, a wrong proposal costs a reviewer a few minutes. Without them, it costs an incident.
In my experience, the gate slows the on-call engineer down less than people expect. Reviewing a clear diagnosis and a ready diff is far quicker than starting from a blank terminal at 3 a.m.
Teams that already run infrastructure through GitOps have most of this in place. A pull request is a sandbox, a merge is an approval gate and a revert is a rollback. Running many clusters from one control plane taught me that the safest automation can only act through the same reviewed path as everyone else. Agents should be held to that rule too.
Containment is a design discipline
VMblog recently ran a piece arguing that kill switches are not enough to control AI agents. I agree, and I would go further. A kill switch is the control you reach for after the agent has already done something you didn’t want. It tells you the system was never designed to be stopped any other way.
Containment works better when it is designed in from the start, along three lines.
What the agent can see. Read access is where most of an agent’s value comes from, and where a lot of the risk hides. Logs and traces can hold customer data, secrets that leaked into output and internal details that should never reach a model’s context. Decide what the agent needs for diagnosis, give it that, and redact the rest before it arrives.
What the agent can touch. Write access should be close to nothing by default. The agent gets its own identity, never a borrowed human login, with permissions scoped to the sandbox where it builds its proposal. The path to production runs through the same approval and deploy pipeline a person would use. If the agent cannot merge, it cannot surprise you in production.
How it is watched. Every tool call, query and proposed change should be recorded as it happens, in a form a person can follow live and audit later. Watch behavior as well as outcomes. An agent that starts querying systems unrelated to the alert, or retries the same failing action in a loop, is telling you something before anything breaks.
None of this is new. It is least privilege, change management and observability, applied to a new kind of actor. Platform teams already know how to do all three. The mistake is assuming an agent is exempt because it is helpful.
The failure mode to plan for
The pattern I expect to see in 2027 goes like this. A team pilots an agent with read access and likes what it sees. The diagnoses are good. Someone points out that the agent could apply the fixes it keeps proposing, and the approval step starts to feel like friction. Broad production access gets granted, because scoping it properly takes work and the pilot went well.
Then the agent meets a situation that looks like one it has seen before and isn’t. It applies a reasonable fix to the wrong problem, at machine speed, possibly in several environments at
once. The postmortem lands where postmortems usually land: the agent had more access than its task required, and nobody could see what it was doing until the damage showed up.
Teams that set boundaries early avoid most of that, and they also move faster. That sounds backwards, so it is worth spelling out. Clear boundaries can be widened on purpose. A class of fix the agent has proposed correctly many times, with a small blast radius and a tested rollback, can move to automatic approval based on evidence rather than optimism. The audit trail is the evidence.
Teams without boundaries have no such record. They cannot tell where the agent was right, and after the first serious incident the usual response is to pull its access entirely. They end up back at zero. Bounded autonomy is what lets autonomy grow.
Before the agent joins the rotation
If your team is putting an agent on call next year, answer these first:
What can the agent read, and what is redacted before it gets there? What identity does it act as, and what is that identity allowed to change? Where does a proposed fix live before approval, and who approves it?
What is the rollback for each kind of change it can propose? Can someone watch what it is doing right now, and replay what it did yesterday? What evidence would you need before letting it act on a class of alert without approval?
My prediction for 2027 is that these questions become part of ordinary platform work, as routine as access reviews and deploy policies. Agents will take on a real share of first response. The teams that come out ahead will be the ones that decided early what their agents are not allowed to do.
##
ABOUT THE AUTHOR
Nitish Mane is a Software Engineer at Philo. He works on cloud infrastructure. Earlier in his career he helped run large-scale cloud infrastructure, including a GitOps control plane managing 34 Kubernetes clusters.
