Most virtualization platforms were tuned for a familiar mix of web tiers, file servers, and relational databases. Graph workloads don’t fit that mix very well, and the mismatch tends to stay hidden until the data gets big. A proof of concept on a modest VM runs fine, the project gets funded, and six months later the same queries are crawling on a cluster that looks healthy by every dashboard the infrastructure team watches. Here’s what actually changes when graph workloads scale on virtualized and hybrid infrastructure, and where it pays to plan ahead.
Why graph workloads stress infrastructure differently
A graph database answers questions by hopping from a vertex to its neighbors, then to their neighbors, and so on for as many hops as the query needs. Each hop looks up a different, largely unpredictable region of memory, so the workload behaves less like a stream of sequential reads and more like millions of tiny, scattered ones. That pattern is brutal on anything that adds latency between the CPU and RAM. It’s also why a graph host can show modest CPU utilization while queries slow down, since the cores are spending much of their time waiting on memory rather than computing.
Scale makes this worse in two ways. The working set grows until it no longer fits comfortably in one VM’s memory, which pushes teams toward bigger VMs or distributed clusters. At the same time, multi-hop queries fan out quickly, so a traversal that touched a few thousand vertices in testing can touch millions in production. Both trends hit the parts of a virtualized stack that are easiest to overlook: NUMA placement, memory policy, and east-west network traffic.
NUMA placement matters more than core count
Modern servers split memory into NUMA nodes, and a CPU reaches its local memory faster than memory attached to another socket. When a VM is larger than a single NUMA node, the hypervisor must spread its vCPUs and memory across nodes, and every remote access adds latency. For a pointer-chasing workload like graph traversal, that penalty shows up on nearly every hop. Sizing graph VMs to fit inside one NUMA node, or exposing a matching virtual NUMA topology to the guest when they can’t fit, is usually the single highest-impact tuning step.
Hardware refreshes can quietly undo that work. Broadcom has documented a Sub-NUMA Clustering performance regression in which a memory-intensive VM moved by vMotion to a new ESXi 8.0.3 host saw query times jump from about four seconds to more than eight minutes. The cause was a BIOS setting enabled by default on the new hardware that split each socket into smaller NUMA domains, and the VM no longer fit inside them. The fix was to disable that setting and confirm the host’s power profile was set to maximum performance. If you’re migrating graph VMs onto new hosts, check the BIOS NUMA configuration before you check anything else.
Treat memory as a hard reservation
Memory overcommitment is one of virtualization’s best features for general-purpose fleets, and one of its worst for graph workloads. Ballooning, host-level swapping, and transparent page sharing all assume that some guest memory is cold, but a busy graph keeps most of its working set hot. Reserve the VM’s full memory allocation, disable ballooning for those guests, and use large pages where the database and guest OS support them. Those settings cost you consolidation density, but they buy you predictable latency, which is the thing your application owners will actually notice.
The economics of that reservation are getting harder, too. Valkey maintainer Madelyn Olson has pointed to DRAM shortages making compression a TCO lever for memory-bound databases, since spare CPU can be traded for a smaller memory footprint without hurting throughput. Graph engines that compress storage on disk and in memory benefit from the same math. When you’re comparing options for a large graph, ask how many bytes per edge each one really needs at your data volume, because that number drives host count more than raw query speed does.
East-west traffic and the distributed cluster
Once a graph outgrows a single VM, it gets partitioned across several, and some traversals will cross partition boundaries on almost every hop. That turns the network between your graph VMs into part of the query path. Keep cluster members on the same low-latency fabric, use anti-affinity rules so two partitions don’t share a host and a failure domain, and give the cluster dedicated vNICs or VLANs so backup jobs and vMotion traffic don’t compete with traversal traffic. Jumbo frames and paravirtual network adapters are worth testing as well, since partition-to-partition messages tend to be small and frequent.
Live migration needs a policy of its own. A vMotion of a large, memory-busy graph VM can take a long time to converge, and it can stall queries on the node being moved while the rest of the cluster waits on it. Many teams pin graph VMs with DRS rules and schedule migrations for maintenance windows rather than letting the scheduler rebalance them during peak hours. If your graph runs on Kubernetes instead, the equivalents are a StatefulSet with pod anti-affinity, Guaranteed QoS pods with integer CPU requests so the static CPU Manager policy can pin cores, and a Topology Manager policy that keeps CPU and memory on the same NUMA node.
Hybrid placement: where the graph should live
The hybrid question has gotten sharper over the past two years. OpenText’s Cloud Repatriation Shift survey found that 67% of respondents have already moved workloads out of public cloud to on-premises, private cloud, or hybrid environments, and 87% plan to move more within two years. Steady, memory-heavy workloads are often the first candidates, because their resource needs are predictable and the premium for elastic capacity buys them little. A large production graph that runs around the clock fits that profile well.
What doesn’t work is stretching a single graph cluster across a WAN link. Traversals that cross partitions will pay the round-trip latency on each hop, and a query that took milliseconds inside one data center can take seconds once a few of those hops leave the building. A better pattern is to keep each cluster inside one site and move data between sites with replication or change streams. You might run the primary graph on premises next to the systems of record and push read replicas or derived subgraphs to the cloud for analytics and model training.
That approach fits the broader shift toward placing compute by latency and data gravity rather than by habit. Derek Gillespie of 365 Data Centers describes a layered edge, regional, and cloud fabric in which routine processing runs close to where data is created and heavier work is escalated only when it needs more compute. Graph workloads map onto that model cleanly: real-time fraud or access checks sit near the transactions they score, while whole-graph analytics can run wherever compute is cheapest that week. The key is deciding which layer owns the authoritative copy and treating every other copy as derived.
What to measure before and after you scale
Generic VM metrics won’t tell you much about graph health, so add a few that will. Track remote NUMA memory access per VM, CPU ready time, and memory balloon and swap activity at the host level. At the network layer, watch latency and retransmits between cluster members rather than just aggregate throughput. Inside the database, log hop depth, vertices touched, and latency percentiles for your heaviest queries, since those numbers move first when infrastructure starts to drag.
Capture a baseline before any hardware refresh, hypervisor upgrade, or migration between environments, then run the same query set afterward. Most of the painful graph incidents on virtualized infrastructure come from changes that look harmless in isolation, like a new BIOS default, a relaxed memory reservation, or a DRS rule that got dropped during a cluster rebuild. A repeatable benchmark turns those surprises into a quick comparison instead of a week of guesswork.
Graph workloads can run very well on virtualized and hybrid infrastructure, but they reward teams that treat them as their own class of tenant. Size VMs to the hardware’s NUMA layout, reserve memory instead of sharing it, protect the east-west network, and keep each cluster inside a single site. Do that, and the graph that impressed everyone in the pilot will keep up once the edge count really starts to climb.
##
ABOUT THE AUTHOR
Danika is a ten-year veteran of the writing industry and is based in Boise, Idaho. Outside of work, she enjoys playing video games and podcasting about The Bachelor.
