Artificial intelligence is transforming businesses, but enterprises are realizing that running AI at scale isn’t as simple as spinning up cloud resources. High costs, GPU inefficiencies, and increasing concerns around data privacy and model governance are driving organizations to rethink their AI infrastructure.
Broadcom’s VMware Cloud Foundation (VCF) is emerging as a cornerstone of Private AI, helping businesses maximize their AI investments while keeping workloads secure and cost-effective. To explore how enterprises are making this shift, VMblog spoke with Tasha Drew, Global Product Engineering Director for VMware Cloud Foundation AI & Advanced Services at Broadcom.
VMblog: Private AI is gaining a lot of momentum. What’s driving enterprises to move away from public cloud AI?
Tasha Drew: Enterprises started experimenting with AI in the cloud because it was the fastest way to get going. But now that they’re moving AI into production, they’re hitting major roadblocks. The biggest challenges we hear from customers are around GPU underutilization, cloud cost overruns, and model governance concerns.
GPUs are expensive, and most organizations aren’t getting the full ROI from them. Meanwhile, AI cloud costs are skyrocketing, and companies are spending way more than expected. And as businesses train AI models on proprietary data, they need stronger security and compliance-something that’s hard to guarantee in a shared cloud environment.
VMblog: Broadcom launched VMware Private AI Foundation with NVIDIA last year. How does this solution fit into the Private AI landscape?
Drew: VMware Private AI Foundation with NVIDIA is a turnkey platform for enterprises that want to run AI securely in their own data centers while still taking advantage of cutting-edge AI tools and models.
It’s built on VMware Cloud Foundation and integrates with NVIDIA AI Enterprise, so companies can fine-tune and run AI models in-house, without relying on public cloud providers, optimize GPU performance with intelligent workload scheduling, and keep AI workloads secure and compliant by controlling where data is processed and stored.
It gives businesses the best of both worlds-the flexibility to innovate with AI while maintaining cost efficiency and security.
VMblog: Let’s talk about GPU utilization. Why is this such a major issue for enterprises, and how does Broadcom solve it?
Drew: GPUs are expensive and hard to get, yet most organizations aren’t fully utilizing the ones they have. The problem comes down to fragmentation and inefficiency. Companies dedicate an entire GPU to a single model, even when that model isn’t fully using the available processing power. This leads to wasted resources and lower overall ROI.
VMware Private AI Foundation with NVIDIA solves this in two ways. First, with intelligent GPU scheduling, instead of letting GPUs sit idle, workloads are dynamically assigned to maximize utilization. And vGPU technology allows organizations to split GPU resources across multiple AI workloads, ensuring that every GPU is working at full capacity.
This means faster AI performance, lower costs, and better efficiency-without requiring additional hardware.
VMblog: AI in the cloud seemed like the obvious choice at first. Why are enterprises running into cost challenges?
Drew: AI cloud costs start off manageable, but they skyrocket as workloads scale.
We see companies launch AI initiatives in the cloud, and then suddenly, they’re spending 3-5x more than expected. The biggest cost drivers are high GPU rental costs (AI-optimized cloud infrastructure is expensive, data egress fees (moving data in and out of cloud platforms adds up fast), and storage and networking costs (AI models generate massive datasets that aren’t cheap to manage in the cloud).
With VMware Private AI Foundation with NVIDIA, enterprises can bring AI back in-house-running workloads on their own cost-controlled infrastructure while still taking advantage of NVIDIA’s AI tools and models.
An IDC study found that 60% of enterprises see on-prem AI as equally or less expensive than public cloud AI. When businesses start doing the math, Private AI becomes a no-brainer. Our customers have been finding a 3-5x TCO decrease.
VMblog: Security and compliance are major concerns for AI workloads. How does VCF help?
Drew: AI security isn’t just about protecting data; it’s about governing the entire AI lifecycle.
Once you train an AI model on proprietary or regulated data, that model inherits that sensitivity. If that model lives in a public cloud, companies lose control over where and how it’s used.
VMware Private AI Foundation with NVIDIA ensures that AI workloads stay secure, compliant, and fully governed by providing on-prem deployment, so companies have full control over their AI models and data, built-in security features like role-based access control (RBAC) and model governance policies, while protecting Private IP, Private Data, and Private Models.
This level of control and compliance is critical for industries like finance, healthcare, and government, where data privacy is non-negotiable.
VMblog: Broadcom and NVIDIA have been working closely on AI. How does this partnership benefit enterprises?
Drew: AI is complex, and most enterprises don’t want to spend years figuring out infrastructure from scratch.
The Broadcom and NVIDIA partnership simplifies AI adoption by bringing together VMware Cloud Foundation for a secure, scalable AI infrastructure, NVIDIA AI Enterprise for access to state-of-the-art AI models and frameworks, and optimized GPU performance, thanks to Broadcom’s AI workload management.
Together, we’re making AI easier, faster, and more cost-effective for enterprises-without the typical infrastructure headaches.
VMblog: Where do you see Private AI heading next?
Drew: AI is moving so fast, and enterprises need infrastructure that can keep up. That’s why we’re focused on a few key things to enable companies to be successful at this accelerated pace, enabling them via an extensible platform approach to deploy new best of breed technologies as they are developed. Smarter GPU scheduling makes AI workloads even more efficient, while enhanced AI model stores will help companies securely manage and deploy AI models. Lastly, we’re investing in advanced cost tracking tools to help businesses optimize AI spending before it becomes a problem.
Private AI is becoming the preferred model for enterprises, and VMware Private AI Foundation with NVIDIA is helping businesses take full control of their AI strategy-from infrastructure to security to cost optimization.
VMblog: What’s your advice for companies exploring Private AI?
Drew: AI isn’t just an experiment anymore-it’s a business-critical function. Companies need to be thinking about long-term AI infrastructure strategies now, not later. If you’re evaluating AI investments, start by asking:
- What business value do I want to get from this investment?
- Are we maximizing our return on investment in AI infrastructure?
- Do we have control over AI costs?
- Are we ensuring security & compliance at every stage?
If the answer to any of these is “I’m not sure,” it’s time to consider Private AI. VMware Private AI Foundation with NVIDIA makes it easier than ever to get started.
##
