VMware Cloud Foundation Is Credible but Limited AI Infrastructure

The practical case for VCF is strongest when an organisation wants to add controlled AI services to systems and data it already operates, without creating a separate GPU and Kubernetes estate. That makes it a plausible platform for focused inference workloads, but not a universal answer for AI experimentation, large-scale model training or weak internal platform capability.

Broadcom is positioning AI as a native part of VMware Cloud Foundation (VCF) rather than leaving it as a partner-led project bolted onto vSphere. VMware Cloud Foundation 9.1 adds Private AI Services, controls for AI-agent tool access and operational visibility into model and infrastructure activity. It also broadens accelerator support across NVIDIA, AMD and Intel hardware. That is a meaningful shift from VMware’s earlier role as the virtualisation layer beneath an AI stack. (news.broadcom.com)

VCF can run AI. That was never really the issue. The harder question is whether its unified private-cloud model makes a particular AI service easier to operate, govern and afford than public-cloud services or a purpose-built Kubernetes and GPU platform. Kubernetes is a platform for orchestrating containerised applications. For defined workloads, VCF may help. For broad experimentation, frontier-model training or teams without solid platform engineering, it usually won’t.

What does VMware mean by private AI?

In Broadcom’s usage, VMware private AI means running models, retrieval systems, AI agents and their supporting data on infrastructure controlled by the customer or an approved service provider, instead of sending every prompt and document to a public AI service. VCF provides the virtualisation, software-defined storage, networking, Kubernetes services, automation and operations layer. NVIDIA, AMD and Intel provide accelerator options; model runtimes and application tooling sit above them.

The appeal is straightforward. A bank, public body, healthcare provider or Channel Islands business handling confidential client material may want an internal assistant that can search policy documents or case files without making a public model endpoint the default path for that data. Keeping inference close to source systems can also reduce awkward network dependencies and make jurisdictional boundaries clearer.

That is not an automatic compliance outcome. Hosting data in a local or UK-controlled environment can help meet residency and contractual requirements, but it does not establish a lawful purpose, minimise data, prevent over-collection or make model decisions explainable. The ICO’s AI guidance is clear that AI can aggravate established security and data-minimisation risks, and that data-protection duties apply throughout the lifecycle, including training data, deployment inputs and outputs. (ico.org.uk)

Which AI workloads can VCF realistically support?

The sensible early targets are inference-led workloads: workloads in which trained models generate, classify or summarise outputs. Examples include internal retrieval-augmented generation (RAG) assistants, document classification, summarisation, software-development helpers, support-agent assistance and controlled agent workflows. RAG retrieves relevant internal material and supplies it as model context, rather than retraining a model on every company document. These workloads benefit from having GPUs, application containers, identity controls and nearby data services under one operating model.

VCF supports virtual machines as well as containers, which matters where useful data and line-of-business systems still live in traditional VM estates. Broadcom positions VCF 9.1 around both workload types and adds management controls for Model Context Protocol (MCP) tools used by agents. MCP is an increasingly common way for an AI application to call approved external tools and services. Controlling that connection matters more than treating every agent as an isolated chatbot. (news.broadcom.com)

GPU virtualisation is part of the economic case. NVIDIA documents support for VCF 9.0 and 9.1 in both vGPU (virtual GPU) and GPU pass-through modes, where a GPU is assigned directly to a virtual machine. VCF 9.1 adds support for MIG-backed multi-vGPU configurations in the relevant NVIDIA stack. MIG, or Multi-Instance GPU, partitions compatible GPUs into separate instances. In some cases, that lets a physical GPU serve suitable workloads rather than being reserved for a single virtual machine. (docs.nvidia.com)

There are limits. Sharing helps with small, variable inference jobs; it is not magic. A latency-sensitive model may need an entire GPU, a larger model may need several, and distributed training depends heavily on GPU interconnects, network design, storage throughput and careful scheduling. GPU profiles create real packing constraints too: NVIDIA notes that profile choice determines how many vGPU-backed VMs can be deployed, and different profiles cannot simply be mixed on one GPU under NVIDIA AI Enterprise software. (docs.nvidia.com)

For organisations training large foundation models from scratch, VCF is unlikely to be the decisive differentiator. Those projects are hardware-and-data-centre programmes first. VCF is better viewed as the operational substrate for enterprise AI services and customised models than as a way to compete with hyperscale AI factories.

Does VCF solve the security and sovereignty problem?

A single infrastructure platform can reduce operational fragmentation; it cannot replace model evaluation, red-teaming, data classification, retention rules, human review for consequential decisions or a proper AI risk assessment.

It improves the infrastructure side. VCF already offers a familiar enterprise control plane: identity integration, workload isolation, segmentation, lifecycle management, operations tooling and recovery options. Broadcom says Private AI Foundation and newer VCF releases add GPU-as-a-Service, air-gap support, model runtime facilities, monitoring and agent-related governance. Air-gapped environments are isolated from external networks. Those are useful building blocks where infrastructure teams do not want a separate GPU island with its own credentials, monitoring and patching process. (news.broadcom.com)

But an AI platform has a larger attack surface than its hypervisor, the software layer that runs virtual machines. The model, container images, vector database, document-ingestion process, prompts, agent permissions, MCP servers and logs all need clear ownership. A segmented workload network does not stop prompt injection in a retrieved document. Role-based access control does not prove that an assistant should be allowed to query a payroll system. An immutable backup is valuable, but it cannot identify an unsafe model response.

This is where the VMware framing needs tempering. A single infrastructure platform can reduce operational fragmentation; it cannot replace model evaluation, red-teaming, data classification, retention rules, human review for consequential decisions or a proper AI risk assessment. Red-teaming tests systems for unsafe or unintended behaviour. For UK-regulated deployments, those controls belong in the project design, not a post-deployment compliance exercise. The ICO specifically points organisations towards security, accountability, lawful processing and individual-rights considerations for AI systems. (ico.org.uk)

What has materially changed in VMware’s private AI strategy?

The change is integration and entitlement, not the discovery that GPUs can run in a VMware environment. VMware Private AI Foundation with NVIDIA was available before VCF 9.0, but Broadcom subsequently said VMware Private AI Services would become a standard component of the VCF 9 subscription rather than a separately positioned AI layer. The 2026 VCF 9.1 release goes further with Private AI Services, MCP access controls and AI-focused dashboarding. (news.broadcom.com)

Enterprises have little appetite for assembling virtualisation, a Kubernetes distribution, GPU software, model serving, data retrieval, identity, observability and an agent framework, then carrying the integration risk themselves. VCF’s best case is simple: fewer seams for operations teams to own.

It is still too early to call VCF a standard enterprise AI platform. Broadcom’s public material describes product capability and intent, not independent proof that VCF has become the dominant production-AI choice. The market remains divided between public-cloud managed services, Kubernetes-native AI platforms, appliance-style GPU systems and private-cloud stacks. A standard will be earned through usable integrations, support quality, cost predictability and customer skills, not a product roadmap.

How do licensing and infrastructure costs affect the case?

This is where many private-AI business cases become less tidy. Broadcom licenses VCF by the physical CPU cores across the ESXi hosts being licensed; ESXi is VMware’s hypervisor. There is a minimum subscription capacity of 16 cores per physical processor. VCF includes an entitlement of 1 TiB (tebibyte) of vSAN capacity per licensed VCF core, while additional vSAN capacity can be purchased where needed. vSAN is VMware’s software-defined storage. (knowledge.broadcom.com)

An AI cluster is not priced only by its GPUs. The bill can include VCF subscription capacity, server CPUs and memory, vSAN or other storage, high-bandwidth networking, power and cooling, hardware support, and NVIDIA software where vGPU capabilities are required. The latter is not a cosmetic detail: NVIDIA’s deployment guidance requires valid vGPU licences for a vGPU installation, supported hardware and specific host BIOS settings such as IOMMU/VT-d, SR-IOV and high-performance profiles. IOMMU/VT-d enables direct device access for virtual machines, while SR-IOV allows supported hardware to expose virtualised functions. (docs.nvidia.com)

Broadcom does not publish a simple universal VCF price list that allows a reliable like-for-like calculation, so prospective buyers should be wary of confident total-cost figures built on list-price folklore. Ask for three-year and five-year models that use realistic GPU utilisation, not an assumption that every accelerator will be busy all day.

When is VCF the sensible choice over public cloud or specialist AI platforms?

VCF is most persuasive for organisations that already have a substantial, well-run VMware Cloud Foundation footprint; have data that genuinely needs to remain inside a controlled operating boundary; expect steady enough inference demand to justify owned GPU capacity; and can support both infrastructure and AI application operations. It is especially credible for internal services combining established VM-based systems with newer containerised applications.

Public cloud remains the better default for uncertain demand, quick proof-of-concepts, access to a wide choice of managed models and large temporary training runs. The economics are not automatically cheaper, but avoiding a major GPU purchase while requirements are still moving can be worth more than a neat sovereignty diagram. Managed model platforms solve different problems from infrastructure control, as the range of AWS AI services such as Bedrock, SageMaker AI and Amazon Q shows.

A specialist Kubernetes or AI platform may fit better where an organisation is already cloud-native, its engineers are comfortable operating GPU schedulers and model-serving stacks directly, or it wants to avoid extending a VMware commercial commitment. That is an architectural decision, not an ideological one. Broader virtualisation value and licensing comparisons need a separate assessment, not a retrofitted AI justification.

Can VMware Cloud Foundation become enterprise AI infrastructure?

Yes, for a defined category of enterprise AI: private, governed inference and agent services that need to sit alongside established applications and data under a familiar operational model. VCF’s direction is more coherent than the old “run it on virtual machines and add a GPU” approach, and bringing Private AI Services into the platform makes it easier to take seriously.

No, if “enterprise AI infrastructure” means one platform that makes every AI workload economical, compliant and easy to operate. It cannot. Start with the data boundary, workload profile and expected utilisation, then test whether VCF removes genuine operational friction. If it does, it is a strong private-AI foundation. If it merely adds another subscription around an underused GPU cluster, it is an expensive way to avoid the harder decisions.