During a technical session at VMware Explore 2025, Chris Wolf, Global Head of AI and Advanced Services at Broadcom, made a statement that cuts through all the AI noise: “What you have now is the equivalent of AWS Bedrock, but I can run it on my on-premises.” That’s not marketing speak, it’s an architectural reality that changes everything about how enterprises should think about AI infrastructure.
Years ago, if you asked a hyperscaler about private AI, they might have laughed at you. “AI is just a cloud use case,” they’d say. Fast forward to 2025, and those same hyperscalers are scrambling to offer their own private AI appliances. But here’s the thing: their track record with on-premises solutions reads like a greatest hits album of unfulfilled promises.
While everyone else talks about AI transformation, Broadcom has been quietly building a true enterprise-grade private AI platform. And the numbers don’t lie. Customers are reporting 3X cost savings compared to public cloud, with one enterprise benchmarking VMware at half the cost of multiple cloud providers.
The Private AI Imperative: Why 2025 Changes Everything
The enterprise AI conversation has shifted dramatically. What started as experimental chatbots and proof-of-concepts has evolved into mission-critical infrastructure decisions. IT leaders are discovering that the real AI challenge isn’t about accessing models, it’s about running them efficiently, securely, and at scale.
The cost economics alone are compelling. When a financial services customer can reduce inference costs by a factor of three, that’s not just operational efficiency, it’s budget reallocation that funds more AI initiatives. If they can optimize their infrastructure that much, it means they have more AI budget to spend in other places.
But cost is just the beginning. Data sovereignty requirements are pulling enterprises back on-premises. Compliance frameworks don’t care how innovative your cloud provider’s AI services are if they can’t meet regulatory standards. The hyperscaler promise of “bring your cloud services on-premises” has consistently failed to deliver. As Wolf put it, “The hyperscaler AI appliance on-prem next year is going to be like a year of VDI” – promising but never quite working as advertised.
Here’s what makes 2025 different: AI workloads demand infrastructure control, not just compute rental. When your GPU clusters are running mission-critical inference workloads 24/7, you need the same operational reliability you expect from your database servers.
What Makes VMware’s Approach Actually Different
Native Integration Beats Bolt-On Solutions Every Time
VMware Private AI Services becoming a standard component of VMware Cloud Foundation 9.0 represents more than feature expansion, it’s architectural philosophy. While competitors add AI capabilities as expensive modules, Broadcom treats AI as infrastructure. “AI is touching every application,” notes Purnima Padmanabhan, VP and GM of Broadcom’s Tanzu Division. “It’s going to be part of everything we do in the IT space.”
This shows up in practical ways. The Spring AI framework integration means your existing Java applications, those “boring Enterprise apps,” as they’re affectionately called, can incorporate AI capabilities without architectural rewrites. Your developers don’t need to learn new languages or frameworks. They use the same Java tools they’ve been using for years.
The difference between native and bolt-on becomes obvious when you look at operations. With VMware Cloud Foundation, your AI workloads get the same high availability, disaster recovery, and lifecycle management as your traditional applications. Try explaining to your CEO why the AI chatbot is down but the payroll system is still running.
Multi-Vendor Hardware Strategy: Choice Without Compromise
Here’s where most vendors stumble: they force you to choose between their preferred hardware partner and operational simplicity. VMware supports AMD, NVIDIA, and even cloud brokering within a single platform. “If a customer builds an AI app and says ‘I’m decommissioning the cluster and bringing in new accelerators,’ they shouldn’t have to refactor the app,” explains Wolf.
This isn’t academic flexibility, it’s business protection. When GPU availability shifts or new accelerators emerge, you want infrastructure that adapts. The alternative is being locked into specific vendors and watching your AI budget disappear into hardware premiums.
The platform’s inference engine, built on open source technologies but hardened for enterprise use, supports everything from commercial models in the cloud to proprietary inference engines that application ISVs build for specialized use cases. You’re not betting on Broadcom’s AI models-you’re betting on their infrastructure to run whatever models make business sense.
The Ecosystem Play: Why Open Beats Vertical Integration
Broadcom’s approach contradicts the current industry trend toward vertical integration. While competitors try to own every layer of the AI stack, Broadcom focuses on what they do best: enterprise infrastructure that works with everything else.
“Are you trying to take over the world?” customers ask, looking at the comprehensive platform capabilities. “The answer is no,” responds the engineering team. Instead, they’re building Windows Server for AI-enough core capability to get started, with open APIs for the ecosystem to build on top.
This philosophy extends to their silicon development. Instead of building networking chips optimized for specific software, they work with dozens of hyperscale customers to understand requirements, then build hardware that serves everyone. When you develop technology for “F1 cars” (hyperscale data centers), you can bring that innovation to the broader market without forcing vendor lock-in.
The Full Stack Reality Check
Application Layer: Making AI Mainstream, Not Special
The real AI challenge for most enterprises isn’t building transformer models, it’s getting AI capabilities into existing applications. VMware’s approach recognizes this reality. The Spring AI framework provides Java developers with familiar tools to build AI-enabled applications. GitOps integration ensures that AI development follows the same deployment pipelines as traditional applications.
Tanzu Data Intelligence addresses the data preparation challenge that kills 30% of AI projects according to Gartner research. Up to 90% of enterprise data is unstructured and largely inaccessible for analysis. The platform provides unified access to multimodal data with millisecond latency, turning data silos into AI-ready resources.
The genius is in the mundane details. Vector databases, streaming data processing, and data lineage tracking are built into the platform. Developers don’t need to become data engineers to build AI applications. They get low-latency access to the data they need, when they need it.
Infrastructure Layer: Purpose-Built for AI Scale
Let’s talk about what AI actually demands from infrastructure. GPUs require roughly 100 times more bandwidth than traditional CPUs. A typical CPU might need 50 gigabits of networking. A modern GPU demands 800 gigabits, with some configurations pushing 3.2 terabytes of I/O per GPU.
This isn’t incremental change – it’s architectural disruption. Traditional enterprise networking, built for CPU-centric workloads, simply can’t handle AI scale. Broadcom’s networking silicon addresses this reality with 100 terabit-per-second switches operating at 250-nanosecond latency. That’s faster than InfiniBand while maintaining Ethernet compatibility.
The power efficiency story is equally compelling. Traditional switches delivering 12.8 terabits of capacity consumed about 20 kilowatts-nearly a full rack’s power budget. Broadcom’s latest silicon delivers 100 terabits (eight times the capacity) while consuming less than 2 kilowatts. When you’re scaling AI infrastructure, that efficiency translates directly to operational costs and data center capacity.
Here’s the part that enterprise architects appreciate: these switches include in-network computing capabilities. While GPUs exchange parameters during model training, the network performs computations on the data as it moves between nodes. It’s not just faster networking, it’s infrastructure that participates in the AI workload.
The Data Challenge: Making Sense of the 90%
The dirty secret of enterprise AI is that most organizations can’t access their own data effectively. Application teams and data teams operate in separate silos, using incompatible tools and processes. Meanwhile, 90% of enterprise data sits in unstructured formats that traditional analytics can’t touch.
Tanzu Data Intelligence approaches this as an architecture problem rather than a data problem. The lakehouse platform provides unified access to structured and unstructured data, with real-time streaming and vectorization capabilities built in. More importantly, it treats data sets as products that can be packaged and delivered to development teams with appropriate governance and security controls.
The platform’s SQL and semantic similarity search capabilities mean developers can query vectorized data using familiar tools. Your Python developers don’t need to become database administrators to build RAG applications. The platform handles the complexity while exposing simple, consistent interfaces.
Strategic Partnerships That Actually Matter
NVIDIA Integration: Coexistence, Not Replacement
The NVIDIA Blackwell integration demonstrates how serious partnerships work in enterprise infrastructure. VMware Cloud Foundation will support the latest GPU architectures, including RTX PRO 6000 Server Edition and B200 GPUs, while preserving core VCF capabilities like vMotion, High Availability, and Distributed Resource Scheduler.
This matters more than you might think. Enterprise workloads don’t exist in isolation. Your AI inference engine needs to coexist with your ERP system, your database clusters, and your legacy applications. The ability to move AI workloads between hosts for maintenance or load balancing, while maintaining the same operational procedures your teams already know, is the difference between AI as infrastructure and AI as science project.
The integration includes support for NVIDIA ConnectX-7 NICs and BlueField-3 400G DPUs with Enhanced DirectPath I/O. These aren’t just performance improvements, they enable GPUDirect RDMA and GPUDirect Storage for high-speed data transfer that’s essential for multi-node AI training.
Microsoft Azure Integration: Hybrid AI Patterns Emerge
Since March 2025, all Azure AI models have been supported on VMware Cloud Foundation. This isn’t just technical compatibility, it’s recognition that enterprise AI follows hybrid patterns. Organizations experiment with public cloud services, identify successful use cases, then move production workloads to private infrastructure for cost and control benefits.
Other major cloud providers are following similar integration paths. The pattern is clear: experimentation happens in the cloud where capacity is on-demand, but production inference workloads move on-premises where costs are predictable and data stays under enterprise control.
The Walmart Validation: When Scale Meets Reality
Walmart’s selection of VMware as a strategic vendor for virtualization software validates the enterprise-scale approach. The world’s largest retailer doesn’t make infrastructure decisions lightly. Their deployment of VMware Cloud Foundation will support enhanced agility, reduced operational complexity, simplified workload portability, and improved security across globally distributed operations.
This isn’t just a customer win, it’s proof that the architecture scales to retail operations that process millions of transactions daily across thousands of locations. When your infrastructure needs to support everything from point-of-sale systems to supply chain optimization algorithms, you need platforms that work reliably at massive scale.
The Technical Differentiators That Matter
Silicon Innovation: When Hardware Meets Software Requirements
Broadcom’s networking silicon represents genuine innovation rather than incremental improvement. The latest switches deliver 100 terabits per second with 250-nanosecond latency while consuming 90% less power than previous generation solutions. These aren’t just impressive numbers-they’re architectural enablers for AI workloads that couldn’t run efficiently on traditional infrastructure.
The silicon development process itself tells a story about enterprise reliability. These chips contain 250 billion transistors-more than a quarter billion logic gates designed in software, tested virtually, then manufactured successfully on the first production run. No respins, no delays, no “almost working” compromises that plague other vendors.
The switches support both copper and optical connections, with co-packaged optics for maximum density. But Broadcom keeps the interfaces open for third-party optical solutions. They’re not trying to own every component-they’re enabling ecosystem innovation while providing the core silicon that makes everything else possible.
Enterprise Operations: AI That Follows IT Rules
VMware Cloud Foundation’s approach to AI operations recognizes that enterprises don’t want separate management planes for AI workloads. The Intelligent Assist capability uses AI to help diagnose and resolve infrastructure issues, but it operates through the same interfaces that administrators already know.
Model Context Protocol support provides governance and security for AI applications that integrate with diverse enterprise tools-Oracle databases, Microsoft SQL Server, ServiceNow, GitHub, Slack, PostgreSQL. Instead of building custom connectors for each integration, developers get standardized methods that maintain security and compliance requirements.
The multi-tenant Models-as-a-Service capability allows secure sharing of AI models between business units while maintaining complete data isolation. This addresses the real enterprise requirement for cost efficiency without compromising security boundaries.
Model Compression: Solving the Inference Cost Problem
Broadcom’s research team has developed state-of-the-art model compression technology that was accepted at top-tier AI conferences. The technique can compress large language models without accuracy loss-a breakthrough that directly addresses inference cost concerns.
The technology isn’t in production yet because they’re optimizing inference response times. While the compression maintains accuracy, there’s currently a performance trade-off that they’re working to eliminate. When ready, this capability will give customers another tool for optimizing AI infrastructure costs without sacrificing model quality.
Looking Forward: What This Means for IT Leaders
The Choice Architecture: Avoiding Tomorrow’s Lock-In
Enterprise infrastructure decisions have 5-10 year consequences. Betting on closed, vertically integrated AI platforms means accepting that today’s vendor choices determine tomorrow’s AI capabilities. The “appliance from OEM plus single AI provider” model might solve immediate requirements, but it creates strategic dependencies that become expensive to change.
VMware’s open, interoperable approach provides choice preservation. APIs compatible with open source projects, support for multiple hardware vendors, and integration with major cloud providers mean that infrastructure investments support business agility rather than constraining it.
The ecosystem approach extends to commercial AI partnerships. Instead of building proprietary models, VMware enables customers to work with specialized AI vendors while maintaining consistent operational procedures. Your infrastructure choice doesn’t determine your AI strategy.
Strategic Implications: Infrastructure as AI Enabler
The shift to AI-native infrastructure changes IT budget allocation and vendor relationships. Instead of buying AI capabilities as separate products, organizations invest in platforms that make AI development and deployment operationally routine.
This approach recognizes that AI isn’t a destination – it’s a capability that enhances existing applications and enables new business processes. The infrastructure needs to support both traditional enterprise workloads and AI applications with the same reliability, security, and operational procedures.
Private AI as infrastructure rather than application means that AI capabilities become available to every development team without requiring specialized skills or separate operational procedures. The platform handles the complexity while exposing familiar interfaces and maintaining enterprise controls.
The Architecture Decision That Changes Everything
VMware isn’t just adding AI features to existing products, they’re architecting infrastructure for an AI-native world. The gap between hyperscaler promises and enterprise realities continues to widen, while appliance-based solutions create new forms of vendor lock-in.
The “private cloud with native AI” approach represents the next phase of enterprise infrastructure evolution. Organizations get the operational control and cost predictability they need for production workloads, with the flexibility to integrate cloud services where they make business sense.
For IT leaders evaluating AI infrastructure strategies, the question isn’t whether to adopt AI, it’s whether to bet on platforms that enable choice and operational consistency, or accept the constraints that come with vendor-specific solutions. The enterprises succeeding with AI at scale are choosing architecture over features, and infrastructure that grows with their requirements rather than limiting them.
The future belongs to organizations that can deploy AI capabilities as routinely as they deploy web applications today. That requires infrastructure built for enterprise realities, not startup experiments. VMware Cloud Foundation delivers that infrastructure, with the silicon, software, and ecosystem partnerships needed to make AI a competitive advantage rather than an operational burden.
##
