VMware, back in the days of early 2001, when there was only VMware ESX Server was built with ONE storage system in mind. A NetApp using NFS. They added support for FibreChannel, iSCSI, etc. but NFS was the main thing for VMware. In many cases it still is! For an average home enthusiast like myself FibreChannel isn’t an option because of price, that means it’s either NFS or iSCSI that’ll be used for storage. Unless you use internal disks in the server itself, but why would you want to do that? Your file server is there for a reason! What are the pros and cons of NFS vs. iSCSI? Here are my thoughts on the matter.

NFS

+

VMware was built for this.

It’s fast when it comes to reading compared to iSCSI, at least in my experience.

The virtual machines are stored as files on the file server and not in an image using VMFS.

-

NFS does sync writes, this tends to be slow. Can be solved by using ZIL, ZFS Intent Log if using ZFS.

Security isn’t the best unless using NFSv4 with authentication, use on a dedicated storage network instead.

 

iSCSI

+

Easier to share between more than one host if using multiple ESXi server.

Can be configured with good security.

-

Slow. At least that’s what my experience tells me.

Not that flexible since the VMs are stored in an image using VMFS, the VMware File System.

 

What to choose?

For me it was an easy choice, I went for NFS. I have used iSCSI before with ESXi and I wasn’t content. It was slow and more or less a pain to backup the machines. With NFS I can just power off the machine and copy the whole thing to another directory on the file server (or a take a snapshot with ZFS). I might add I did use iSCSI under Solaris with the COMSTAR iSCSI target, haven’t tried it under FreeBSD which I’m currently using.

Which ever one you go for I recommend using a dedicated network for storage. This is mostly for security reasons, you don’t want anyone to mess with the storage traffic. Storage also tends to use quite a bit of bandwidth, you probably don’t want this on your LAN. Since I only have two machines, the file server and the ESXi server, I use a single cable between them. No need for a network switch.

Normal load on my ZFS storage system

Normal load on my ZFS storage system

As you can see above I tend to get a lot of reads, I’d say 99% of my storage I/O are read requests.