Zesty announced support for NVIDIA’s CUDA-native GPU fractioning. Following NVIDIA’s open-source release of Run:ai as part of the NVIDIA DSX AI Factory Platform, Zesty’s Autonomous GPU Resource Management adds the autonomous layer that continuously decides GPU allocation and workload placement as AI workload demand fluctuates. With these autonomous capabilities, teams can improve GPU utilization, run more workloads on existing infrastructure, reduce the need for excess GPU capacity, and lower infrastructure costs while protecting application performance.
66% of organizations hosting generative AI models run some or all of their inference on Kubernetes, and GPU capacity for those workloads is expensive and often constrained. Teams have traditionally allocated entire GPUs to individual workloads or shared them using time-slicing, NVIDIA Multi-Process Service (MPS), or NVIDIA Multi-Instance GPU (MIG). MIG divides each GPU into hardware-based partitions of predefined sizes, and resizing a partition requires clearing the workloads running on it. Because of that, teams often lock in GPU allocations ahead of time. Allocations set in advance cannot keep pace with dynamic AI workloads and real-time changes in demand. As a result, GPU memory and compute end up stranded, and teams buy more GPUs even while existing capacity sits unused.
NVIDIA’s recent open-source release of Run:ai brings CUDA-native GPU fractioning to Kubernetes through the GPU Fractioning Operator. The fractioning mechanism defines how workloads can divide a GPU but does not determine how to allocate that capacity. Deciding which workloads share each GPU, how much memory each one receives, and how those allocations should change as demand rises and falls requires a separate decision layer.
Zesty provides that layer. Its Autonomous GPU Resource Management continuously determines which workloads should share each GPU, how many can safely run together, how much GPU memory each one receives, how compute is shared, and when those allocations should increase or decrease. Decisions draw on actual GPU memory usage and utilization, historical workload behavior, demand peaks and spikes, latency, out-of-memory (OOM) history and risk, workload requests and limits, workload type and workload-specific policies. Every change stays within the performance and availability policies teams set for each workload.
“GPUs are among the most expensive and constrained resources infrastructure teams manage today. They’re difficult to secure, costly to add, and shared by AI workloads whose needs change constantly. Now NVIDIA has opened GPU fractioning to the ecosystem, and Zesty is making their allocation intelligent and autonomous,” said Pini Ben Nahum, VP R&D at Zesty. “Autonomous GPU Resource Management adjusts GPU memory, compute, and workload placement as each workload’s needs rise and fall. Teams can pack more onto existing hardware, reducing new GPU purchases and the cost of running AI in production without giving up performance quality.”
The capability deploys as a lightweight agent in the Kubernetes cluster and works alongside NVIDIA Run:ai and the Kubernetes scheduler rather than replacing the scheduler. It requires no application code changes, no model changes, and no manual Kubernetes manifest changes, and initial deployment requires no workload restarts. Autonomous GPU Resource Management is available today as part of the Zesty platform.
