Virtualization Technology News and Information
VMware vSphere: The CPU Scheduler in VMware ESX 4.1

With the release of VMware vSphere 4.1, VMware is also updating and creating a number of new technical whitepapers around the new platform.  Check out this new paper, "The CPU Scheduler in VMware ESX 4.1," which is 22 pages and discusses the role of the CPU scheduler and provides an in-depth analysis of the design changes found in ESX 4.1.  And make sure to check out the performance comparison between ESX 3.5 and ESX 4. 

The introduction of the paper reads: 

The CPU scheduler in VMware vSphere™ 4.x (ESX™ 4.x) is crucial to providing good performance in a consolidated environment. Because most modern processors are equipped with multiple cores per processor, or chip multiprocessor (CMP) architecture, it is easy to build a system with tens of cores running hundreds of virtual machines. In such a large system, allocating CPU resources efficiently and fairly is critical.

In ESX 4, there are some significant changes to the CPU scheduler—for better performance and scalability. This paper describes these changes and their impacts on performance. It also provides certain details of the CPU scheduling algorithms.

This paper is updated to reflect the changes in the ESX 4.1 CPU scheduler. Sections 3.4 and 4.4 discuss wide-VM NUMA support and its performance impact.

It is assumed that readers are familiar with virtualization using VMware® ESX™ and already know the common concepts of the CPU scheduler. However, it is also strongly recommended to read the vSphere Resource Management Guide [1], because this paper frequently refers to it.

In this paper, a pCPU denotes a physical CPU; a vCPU denotes a virtual CPU. The former refers to a physical core on a system, and the latter refers to a virtual machine’s virtual processor. A host refers to a vSphere server that hosts virtual machines; a guest refers to a virtual machine on a host. In describing CMP architecture, a socket or a processor package can be used to denote a chip that has multiple cores.

Like a process, a vCPU can be in one of the following states: In running state, a vCPU executes on a pCPU. In ready state, it is runnable but waiting in a queue. In wait state, it is blocking on a resource. An idle vCPU can enter wait_idle, a special wait state, which does not depend on a resource. The idle vCPU changes its state from wait_idle to ready when interrupted.

The rest of this paper is organized as follows: Section 2 describes the key features of the CPU scheduler, which comprise ESX 4.0/4.1 and ESX 3.5. The section applies, in most part, to older versions of ESX as well. Section 3 describes the major changes introduced by ESX 4.0/4.1 and discusses their performance impact qualitatively. Section 4 presents experimental results, which show the impact of the changes described in this paper. Finally, Section 5 summarizes this paper.

Download this paper, here.

Published Thursday, July 22, 2010 6:21 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<July 2010>