VMware RAID 5 vs 10 and stripe size

I've been setting up a new development VMware server, and I've learned a few things about RAID performance. Our server is a brand new 3GHz Quad core Xeon, with 10 gigs of RAM. We initially set it up with 5 disks, 4 for RAID 5, and the fifth as a spare.

VMware virtual machines tend to be huge, especially if you're pre-allocating the disks. In my case, I've pre-allocated them to have 15GB hard drives. I set up a couple on my laptop, and then tried to copy them up to the development server.

Much to my surprise, the write performance was horrible. Copying one large file effectively brings the server to a halt, and the copy isn't really that fast to begin with.

Upon further investigation, I found out that RAID 5 write performance is less than desirable. Write performance is pretty important when you're moving around huge files.

RAID 5

This morning we decided to convert the array to RAID 10, sometimes known as raid 1+0. Write performance is great, and read performance is just as good as RAID 5. We also increased the NTFS cluster size to 64KB, to better match the types of files we're storing (big ones).

 

image

The results have been excellent. I can now copy a virtual machine in 10 minutes instead of 30+. The only disadvantage is that I had to lose 143GB (1 drive in the array).

If you're setting up a VMware server, I recommend:

  • Set the NTFS cluster size high (64k), and match that to your stripe size
  • Use RAID 10
Technorati Tags: ,

posted @ Friday, October 26, 2007 10:36 AM

Print

Comments on this entry:

# re: VMware RAID 5 vs 10 and stripe size

Left by Jafin at 12/16/2007 6:31 PM
Gravatar
Thanks for the article.

I found this paper on performance when changing cluster size from 4k to 64k. He notes in the benchmarks that there appears to be little difference in performance http://www.wilson-it.com/resources/vmwarentfsclusters.pdf

I totally agree RAID5 for VM's bites, we had very poor performance on a RAID5 iscsi as a storage for vm disks.

# re: VMware RAID 5 vs 10 and stripe size

Left by CaB at 6/23/2008 10:50 AM
Gravatar
From what I understand, the controller implementation can have a huge impact on write performance on RAID5, particularly in the case of writing a contiguous block. The write performance can be improved by caching the writes at the controller and calculating the parity for the entire stripe before writing. Unfortunately, this performance-enhancing feature isn't as prevalent as it (i believe) should be. If your write performance is that so terrible that it turned you off of RAID5, I'm guessing your controller didn't have that feature.

Your comment:



 (will not be displayed)


 
 
 
Please add 2 and 3 and type the answer here:
 

Live Comment Preview: