iSCSI vs. NFS on vSphere 5.
Added by Bee Gee about 1 year ago
I've been running the product for some time on NFS, but would love to go to iSCSI to take advantage of VAAI.
There seems to be an abundance of negative press surrounding iSCSI in the forums so I was wondering what the "gotchas" are with using iSCSI and vSphere 5.
I keep things very very simple. I'll be snapshotting once daily and cloning VMs a lot.
My hardware is a Dell c2100 with 22x7200rpm disks, and 4 SSDs for zil/l2arc. I've currently got a 4 port 1GbE NIC which I intend to replace with 2 port 10GbE ASAP.
Replies
RE: iSCSI vs. NFS on vSphere 5. - Added by Linda Kateley about 1 year ago
Brett,
You've come into this decision at exactly the right time. As long as you get the patch for vsphere 5.0.2, it is now reported to work(i have read about a dozen blogs). It would be nice to have validation on this from the community. Also make sure your nexentastor is also fully patched.
linda
RE: iSCSI vs. NFS on vSphere 5. - Added by Jeff Gibson about 1 year ago
Is this patch available from VMWare's site or only through the update manager? Is this something you have to contact their support for?
RE: iSCSI vs. NFS on vSphere 5. - Added by Jeff Gibson about 1 year ago
That's good to know the required command line, but my question is on the VMWare version. I'm not seeing anything past 5.0.0 and corresponds to build 469512. What build is equal to 5.0.2 and is that only available to beta testers?
RE: iSCSI vs. NFS on vSphere 5. - Added by Ashley Watson about 1 year ago
Looks to me that what is being referred to is the vSphere 5.0 update 1 release which is not currently available in GA. A quick google would indicate quite a few fixes at the network and storage layers.
http://blogs.vmware.com/vsphere/2011/09/vaai-thin-provisioning-block-reclaimunmap-issue.html
" Hi Schistad,
When vSphere 5.0U1 releases, we will hopefully have a solution to this issue which will work for all interested parties.
I can't really say anymore than that at this time.
Posted by: Chogan | 01/27/2012 at 07:10 AM "
RE: iSCSI vs. NFS on vSphere 5. - Added by Bee Gee 10 months ago
Linda Kateley wrote:
Brett,
You've come into this decision at exactly the right time. As long as you get the patch for vsphere 5.0.2, it is now reported to work(i have read about a dozen blogs). It would be nice to have validation on this from the community. Also make sure your nexentastor is also fully patched.
linda
Linda,
Please follow up. I can not find anything confirmed stating that VAAI works with NFS datastores running off of Nexenta NFS shares on ESXi 5u1+ hosts now.
RE: iSCSI vs. NFS on vSphere 5. - Added by David Bond 10 months ago
Linda,
Please follow up. I can not find anything confirmed stating that VAAI works with NFS datastores running off of Nexenta NFS shares on ESXi 5u1+ hosts now.
I believe Linda would not be referring to NFS, but to the bug with iSCSI and VAAI, causing hangs. As far as I know nexenta have yet to implement VAAI with NFS.
RE: iSCSI vs. NFS on vSphere 5. - Added by Tommy Scherer 10 months ago
That is correct VAAI has not been implemented on NFS. 3.1.3 should have fixed all of the iSCSI locking issues. I have a customer who we just fully patched a Nexenta Cluster and ESXi Cluster to test with. So far it looks like all of the VAAI functions work with out the dreaded hangs.
RE: iSCSI vs. NFS on vSphere 5. - Added by Bee Gee 10 months ago
Tommy,
Thanks for the head's up on 3.1.3's successful fix of the iSCSI issues.
How bad is the patch process?
Linda,
Any word on when support for VAAI on ESXi host NFS datastores will be supported by Nexenta? Is it on the roadmap?
And while we're on the topic, any support planned for VASA?
Thanks!
RE: iSCSI vs. NFS on vSphere 5. - Added by Tommy Scherer 10 months ago
The process is super easy. upgrade VMware to esxi 5.0 u1 and upgrade Nexentastor to 3.1.3. The way VMware fixed the issue was with a band aid. The biggest issue was SCSI unmap to reclaim blocks. they have added a new command to esxi 5.0 u1 called "vmfstools" with a -y to reclaim blocks on a thin volume. It is a manual reclamation rather than the automated process the originally released. You want to be sure to reclaim the space during down times or off peak hours due to excessive I/O on the VMs datastore (latency will go through the roof)
RE: iSCSI vs. NFS on vSphere 5. - Added by FREDY . 10 months ago
So is SCSI unmap fixed now on 3.1.3 ? Do I understand it right that is safe to enable it on ESXi 5.0 U1. Who here have it enabled and working fine for a while ? I think people reported problems with SCSI UNMAP when doing lots of Storage VMotion.
RE: iSCSI vs. NFS on vSphere 5. - Added by FREDY . 10 months ago
Also wanted to say that if you have 10Gb Network cards and a ZIL disk go for NFS and forget VAAI totally. The flexibility and simplicity you get make it worth. When they implement VAAI for NFS will be just one bonus, but NFS+10Gb+ZIL is just as good or better then iSCSI+VAAI.
RE: iSCSI vs. NFS on vSphere 5. - Added by Tommy Scherer 10 months ago
SCSI unmap is fixed on the VMware side.... I think you need to use the unmap command with cation like 2:00AM on a weekend to prevent timeouts due to space reclamation from two file systems(VMFS 5 and ZFS).
I Agree with Freddy the if you have 10GbE and ZIL you should run NFS ! Even if you have GbE i have gotten pretty good throughput with LACP or IPMP.
RE: iSCSI vs. NFS on vSphere 5. - Added by Edmund White 9 months ago
FREDY . wrote:
Also wanted to say that if you have 10Gb Network cards and a ZIL disk go for NFS and forget VAAI totally. The flexibility and simplicity you get make it worth. When they implement VAAI for NFS will be just one bonus, but NFS+10Gb+ZIL is just as good or better then iSCSI+VAAI.
How is your NFS+10GbE network laid out? Single links between the ESXi hosts and the NexentaStor box? Multiple? Are you doing any sort of load balancing or link resiliency?
RE: iSCSI vs. NFS on vSphere 5. - Added by Bee Gee 9 months ago
I've got Dell c2100s with 2x Intel x520-DA2 cards (one mezzanine and one PCI-E) running 10GbE over Dell 8024F switches. Connects are made with TwinAX/DAC(direct attached copper) 10GbE cables.
The x520-DA2s are used in my ESXi hosts as well and each card has 2x 10GbE ports. Each port plugs into a different 8024F switch and the Nexenta runs 4 x 10GbE ports aggregated into a single virtual port.
This all maps to one massive datastore in ESXi for each Nexenta box.
Then I've got some small appliances that I use for backups and I just use clones to back up after some nightmare moments with CBT based backup solutions.
RE: iSCSI vs. NFS on vSphere 5. - Added by Bee Gee 9 months ago
When I posted this, I think I was still on 1GbE and it was a nightmare... 6 hours to clone a kit that has to be cloned weekly.
That same kit on the same hardware (+10GbE) takes an hour to clone.