I’m running VMware ESXi to virtualize my computers and FreeBSD 9.0 with ZFS as backend storage. I’ve done some tests a couple of years ago telling me to always choose NFS over iSCSI for virtual machine storage. Now, is this the best solution for performance? I had to test this, totally unscientific but still worth reading I hope.
I did two different tests, one with bonnie++ and one with dd. Each test was performed inside a virtual machine (Ubuntu 12.04 LTS 64 bit) running off iSCSI storage and one stored on NFS. They were both stored on the same storage pool. At last the test was performed directly on the FreeBSD backend server to see the potential maximum. This isn’t true though as I was limited by my 1 Gb/s Ethernet connecting the ESXi machine and the FreeBSD server.
I just ran bonnie++ with standard settings, no tweaking at all. Here are the results.
The second test was done with dd. I used dd if=/dev/zero of=./test bs=`expr 1024 \* 1024` count=4096 to create a 4 GB file just measuring the write performance.
As you can see there was a slight advantage for NFS, although not a huge one. In my opinion, the fact that the virtual machines are stored as files on the FreeBSD servers filesystem in NFS, is a giant reason for NFS as the first choice. You can take backup of your machines with ease and won’t have to use VMFS at all.