AMD Best Practices Series. A Contributed Series by AMD.
Understanding the Bigger Picture of VMmark Benchmarks is written by Tim Mueting, Product Marketing Manager at AMD
At AMD, we constantly hear from customers about different issues they are grappling with. It's not to say that all customers face the same issues, but we certainly see a few common themes come up time and time again. Because of that, we thought it would be useful to start a "Best Practices" blog series that helps customers navigate these issues and understand the nuances of the server market. Let's start first with virtualization benchmarks. There's a lot more going on than what we see at the surface.
For example, the VMmark score of HP's two-socket DL 385 G7 VMmark score (30.96) is evidence of the value of the architectural enhancements that the AMD OpteronTM 6100 Series processors bring to the table for virtualization workloads. This score certainly is impressive and highlights the performance benefits of "real cores" and additional memory channels (33 % more per processor than competitive 2P solutions). The DL 385 G7 VMmark score slightly edges out the DL585 G6 - HP's 24 core, 4P "Istanbul" processor-based server - which received a score of 29.95. In addition, the result of the DL 385 G7 is within 15% of Cisco's UBS 250 M2 top score and within 10% of Fujitsu's BX922 X2 score. Both of these machines are based on Intel's 3.33GHz, 130W Xeon model X5680 processors with 192GB of memory. In comparison, the HP DL 385 G7 runs on a 2.3 GHz, 130 W AMD Opteron model 6176 SE with 128GB of memory.
But, like I said, this doesn't quite tell the entire story. Take price, for instance (You can find AMD pricing here and Intel pricing here, as of May 13, 2010). In this case, while we don't have the retail prices of either the Cisco or Fujitsu systems, we can at least compare the price of the processors. As you can see from the chart above, both of the Cisco and Fujitsu systems come with a premium of approximately $554 for the processors (2 processors x $277 more per processor).
Another issue to consider is that of memory. The Fujitsu and Cisco systems use 33% more memory than the DL 385 G7 (192 GB for the Cisco and Fujitsu systems vs. 128 GB for the HP system. Moreover, the memory used by the Fujitsu server maxes out the 12 available DIMM slots with 16GB DIMMs (a product which is not yet widely available, and can go for around $1500 per DIMM).
It's also interesting to note how the Cisco and Fujitsu server scores varied even though they were based on the same processor and amount of memory. I'm only speculating, but it may have something to do with Cisco's proprietary Extended Memory Technology.
The folks at VMware have spent considerable time and effort in developing a virtualization benchmark that represents the relative performance of a server platform. But as with many benchmarks, it doesn't always represent real world scenarios. VMmark, for example, was developed in 2006 when virtualization was focused on less robust applications. We should also recognize that:
- VMmark VMs are very lightly loaded with minimal I/O.
- The VMs in VMmark are not very robust - using only 1 or 2 vCPUs per VM.
- The VMs in VMmark use very little memory - a maximum of 2GB for each VM.
- The goal of VMmark is to run as many virtual machines as possible on a single physical server.
- The individual workloads may not reflect what is typically running in the enterprise, nor do they stress the hypervisor.
- Customers still only run an average of 12 to 15 VMs on a single server as opposed to the 126 VMs used for the DL385 G7 publication. There have been many studies on this topic such as this one by SourceDataCenter, "The data center purchasing intentions."
I'm sure a variety of OEMs will be posting more results very soon, but until then I think this gives us a great picture of where we are today. Based on the results of the DL 385 G7, servers using the new AMD Opteron 6100 Series processors can offer customers up to twice the virtualization performance as the previous generation processors (codenamed "Istanbul") and one of the best price/performance values in the marketplace today.
Does this help answer your questions about measuring virtualization performance? What other questions can we answer?
Tim Mueting is a Product Marketing Manager at AMD. His postings are his own opinions and may not represent AMD's positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.
The AMD Virtualization Blog can be found here.
####
*The results stated above reflect results published on http://www.vmware.com/products/vmmark/results.html, as of April 27, 2010. The comparison presented above is based on the two-socket servers using AMD Opteron™ processors Model 6176 SE and Intel Xeon processors Model X5680. For the latest results, visit http://www.vmware.com/products/vmmark/results.html.
*Configuration Information:
2 x AMD Opteron™processors Model 6176 SE (12-core 2.3GHz) in HP ProLiant DL385 G7 server, 128GB (16 x 8GB DDR3-1333) memory , VMware® ESX 4.0 Update 1, VMmark™ V1.1.1
http://www.vmware.com/files/pdf/vmmark/VMmark-HP-2010-04-20-DL385G7.pdf
2 x Intel Xeon processors Model X5680 (6-core 3.33GHz 130W TDP) in Cisco UCS B250 M2 server, 192GB (48 x 4GB DDR3-1333) memory using Cisco Extended Memory Technology, VMware® ESX 4.0 Update 1, VMmark™ V1.1.1
http://www.vmware.com/files/pdf/vmmark/VMmark-Cisco-2010-04-06-UCS-B250-M2.pdf
Cisco memory prices can be found at http://www.cisco.com/en/US/prod/collateral/ps10265/ps10280/ps10300/white_paper_c11-525300.pdf
2 x Intel Xeon processors Model X5680 (6-core 3.33GHz 130W TDP) in Fujitsu BX922 S2 server, 192GB (12 x 16GB DDR3-1066) memory, VMware® ESX 4.0 Update 1, VMmark™ V1.1.1
http://www.vmware.com/files/pdf/vmmark/VMmark-Fujitsu-2010-04-06-BX922S2.pdf
*Fujitsu memory prices can be found at http://www.stockinthechannel.com/p/S26361-F3284-L525-16GB-DDR3-1066MHz-Memory-Modul-Fujitsu-memory-modules/1698619.aspx