The cloud computing industry learned a hard lesson over the past several years — at scale, the economics of public cloud can quietly erode enterprise value in ways that aren’t immediately obvious. Now, according to Sirish Raghuram, Co-founder of Platform9, AI is setting up to repeat that same pattern, only far more aggressively. Drawing on the landmark 2021 Andreessen Horowitz essay that identified what became known as the Trillion Dollar Paradox, Raghuram warns that AI inference workloads are 3–5 times more expensive per hour than traditional cloud compute, while the rise of agentic AI is simultaneously driving 5–20 times more compute workload per developer — a compounding effect that makes the original cloud cost problem look modest by comparison.
Raghuram believes enterprises have, at most, two years before the upside-down economics of AI demand serious architectural reckoning. Three forces are driving the timeline: the opacity and volatility of token-based pricing, which he characterizes as an emerging supply chain risk; parabolic price inflation for memory and cloud server hardware; and the financial pressure on AI vendors to justify the more than $10 trillion in combined valuations their sector now commands. In this VMblog Q&A, Raghuram shares what enterprises are getting right — and where they are going dangerously wrong — as they navigate the infrastructure decisions that will define their AI competitiveness for years to come.
VMblog: What is the great AI paradox on the horizon?
Sirish Raghuram: The emerging paradox is that AI economics are on track to recreate — at 10× scale — the same cost‑gravity problem that hit cloud computing over the last few years, coined the Trillion Dollar Paradox – for those familiar.
For those unfamiliar with this, in 2021, Sarah Wang and Martin Casado at Andreessen Horowitz published a now well referenced essay titled The Cost of Cloud, a Trillion Dollar Paradox. Their argument was deceptively simple: somewhere between $500 billion and $1 trillion of enterprise market capitalization was being quietly suppressed by the cost of hyperscale cloud at scale. Dropbox, they pointed out, had saved $75 million over two years by repatriating workloads off a public cloud.
The piece did not argue that the public cloud was a mistake. It argued that the received wisdom says “you’recrazy if you don’t start in the cloud, you’re crazy if you stay on it.”
AI is now setting up the same pattern, but in an even more frenzied cycle.With tokens as the new compute-hour, GPU‑driven inference workloads are 3–5× more expensive per hour due to the higher component cost, demand-supply imbalances, and higher cost of power. At the same time, increasing use of agents is creating 5–20× more compute‑workload per developer. It’s the same problem, but 15-100x worse.
VMblog: When and why do you think this is going to hit?
Raghuram: I predict the upside-down economics of AI is going to hit within the next two years, maybe sooner judging by some of the parabolic changes we are seeing in the market… I think there are going to be three major causes of this (even beyond economics), but I will list that here too.
- Tokenomics: Token/consumption pricing is volatile, opaque, and emerging as the new wildcard in infrastructure pricing. Every employee that successfully became AI-native, is now highly dependent on an opaque token price, becoming a variable cost that can be repriced overnight by one of a handful of companies in San Francisco.
- Memory & Cloud Server Pricing: We’re seeing parabolic, hyper-inflation like price changes for commodities such as memory and cloud servers. It’s not just the big providers, even mid-sized bare metal cloud server providers have reportedly doubled the price of common configurations in the last 60 days. This is not normal, and not sustainable.
- Financing consequences: Private investors have driven up valuations of leading AI companies to some of the highest in the world. The value of just a handful of leading AI players now exceeds $10T, i.e. ~15% of the entire stock market capitalization in the US. With US GDP only growing 3% a year, there isn’t a lot of room left for the frenzied growth rates of the past, so this sector will have to raise prices and profitability to sustain their lofty valuations.
The question is who is going to incur these costs? The answer: the consumer.
VMblog: How are you seeing enterprises address this now and where do they go wrong?
Raghuram: I think this is a very dynamic situation, with some of the most savvy early adopters of AI realizing that AI is literally doubling the cost of each developer; yet a lot of others are still early in their adoption maturity. No matter where you are in the adoption journey, my recommendations are:
Treat token pricing as a critical supply chain risk: As explained earlier, AI users are increasingly complaining about the opacity of their AI tools. Many users are suddenly realizing they are burning through credits much faster than expected. Executives are realizing that developers are using credits that exceed their annual salaries. All of these point to the inescapable reality that token pricing is a critical, opaque, external supply chain risk to the modern enterprise.
Measure token efficiency: As enterprises move beyond early AI pilots to truly realizing faster innovation and higher productivity with AI, they need to measure the ROI of their AI toolchain. How much faster are they innovating, at what additional cost? How much additional topline or bottomline results did that deliver, and what is the payback period? If you cannot answer these questions, you don’t know what your efficiency metrics are, and you need to establish those first.
Pilot private inference, with open-source models, to improve token efficiency: Leading open-source models have greatly narrowed the gap vs frontier closed source AI models. In fact, when deepseek first came out, it shocked the world with the efficiency with which it was developed, and closed-source AI leaders had to scramble to adopt its training and development approach. For any business that is concerned about the emerging supply chain risk from AI, developing a credible, open-source based architecture to deploy private AI inference is a critical investment.
Where they go wrong is assuming this is a temporary pricing anomaly or a vendor‑negotiation problem and moving back to the public cloud too quickly.
VMblog: How do private clouds play a role now and in the future in helping address this paradox?
Raghuram: Private clouds have lagged public clouds in the early phases of AI adoption. However, as the cost economics of AI infrastructure start to emerge, businesses will need to adopt a credible private cloud strategy to continue leveraging AI at scale. Why? Because at scale, the economics of private, owned or leased hardware, that is depreciated over time, are impossible to ignore. For any business with >500 employees, the incremental labor cost of managing server hosting is virtually zero; and certainly more efficient than the endless FinOps hand-wringing about public cloud costs.
Moreover, private AI architectures allow data to be processed more efficiently and securely. For example, a large amount of sensor data being produced in a factory, or oil rig, or a hospital can be analyzed and reduced into concise vector data at the edge.
VMblog: What architecture considerations do large enterprises and MSPs need to think about when selecting the infrastructure for their AI workloads?
Raghuram: Key considerations:
Portability: Avoid trading hyperscaler lock‑in for private‑cloud lock‑in. The operating model must be portable across vendors and hardware.
Heterogeneous GPU support: Architect for NVIDIA today, but ensure readiness for AMD, custom silicon, and inference‑optimized chips.
Latency‑sensitive design: AI agents, copilots, and real‑time workflows degrade sharply with WAN latency. Architect for sub‑50ms round‑trip performance.
Regulatory zoning: Build infrastructure that can enforce workload‑level jurisdictional boundaries.
Cost‑predictability: Token‑metered workloads require deterministic cost envelopes. Private inference provides that; architecture must support it.
The winning architectures are hybrid, portable, GPU‑agnostic, and sovereignty‑aware.
##
