3.1.3 will be based on our code base. The 4.0 is planned to be an illumos build.

We had a number of late bindings for 3.1.3 which needed additional testing. We are still hoping to see it very soon.

It has been reported in the community that the vmxnet3 is already in the current 3.1.2.

Hi Ashley,

I tested both Openindiana and Illumian and I would high recomend you stick with Openindiana for the momment. Yes Illumian has the apt-get tool but I don't think it is mature yet to be used in production (I had a couple of issues instaling things from its repository and warnings during boot), while Openindiana+napp-it is fine, and as you know both are based on IllumOS. I also managed to install VMware Tools on Openindiana and it worked fine for me.

I have not much played with plain illumos lately, limitedly OpenIndiana and illumian - and on hardware not vmware, but had generally good impression. Additionally to Linda's statement I can add that NexentaStor 3.x is the current result (and likely to stay as such) of OpenSolaris 134b + Ubuntu-hardy based NexentaCore 3.x (R.I.P.) + backport from illumos + proprio patches

NexentaStor 4.x will be based on illumos (Nexenta's branch is at at https://bitbucket.org/nexenta/illumos-nexenta/), a small userland packaged, all DEB-packaged (maybe something partly using illumos-userland? definitely not Ubuntu/Debian + proprio patches + their Management SW on top.

The beta program for v4 is active but I haven't received beta code so far - still waiting.

Linda Kateley wrote:

3.1.3 will be based on our code base. The 4.0 is planned to be an illumos build.

We had a number of late bindings for 3.1.3 which needed additional testing. We are still hoping to see it very soon.

It has been reported in the community that the vmxnet3 is already in the current 3.1.2.

VMWare tools can be installed on Nexenta 3.1.2 but vmxnet3 is NOT functional out of the box according to all tests that I have done... I have not seen anyone demonstrating a vmxnet3 NIC working in Nexenta 3.1.2 (modified or otherwise), and this issue has been brought up several times on these forums. Can you please provide a reference?

To restate Ashley's main question: Can you post a list of BUG ID's including description and summaries that are absolutely going to be included in the 3.1.3 release? I understand that the software is a moving target and that there might be a few bug missing from the list, but I am sure that you can state at least 40+ of the ~50 bugs that are being resolved. I am sure someone is compiling this information for the changelog so I assume the process won't be too painful. Is this site used by nexenta developers or is it deprecated (http://nexentastor.org/projects/site/issues) ?

My perspective on Ashley's last question: I have tested napp-it on top of the latest openindiana and got vmxnet3 working easily. Performance was good, stability was good... my gut says it is comparable to Nexenta (or better?) and I confirmed it doesn't have several annoying quirks like CPU utilization. However my testing wasn't extensive or long-lasting because frankly i'm not a fan of napp-it's interface and IMO the openindiana OS still lacks polish (it REALLY pisses me off how hard it is to set a static IP and resolve.conf file in that 'server' OS).

Semi-related: On Nexenta 3.1.2 I noticed that my zil is never used when accessing iscsi targets, but as soon as i switch over to NFS the zil is used. This improves write performance of course, and is especially noticeable on volumes with a large amount of data. Note: iometer 2006 tests against esxi datastores attached via iscsi/nfs seem to have about equal performance when the zfs pool is empty, BUT when the volume capacity was 50% consumed nfs maintained performance but iscsi tanked. I discovered this after i concluded my tests with napp-it and never determined if this is a yet another nexenta bug or if it is something else. I'd be curious if you see something similar.

Hi Matt,

Probably you Zil isn't used with your iscsi target because you have write back enabled on the zvols, disable it and it will start using your Zil. When using NFS instead it is like that by default. It's good to have the write back enabled on the zvols if you don't have zil disks but at the cost of loosing a few seconds of data if power goes down.

Yes write-back enabled cache writes into ram and when it is turned off they will be cached into the zil device.

By the way Linda,

NFS by default will try to use Zil and if not available will work do sync to the pool. How do we enable write-back for NFS(async basically) so it can acknowledge the writes in memory as the zvols do ?

Set the pool(or folder) property to sync=disabled

Matt Van Mater wrote:

VMWare tools can be installed on Nexenta 3.1.2 but vmxnet3 is NOT functional out of the box according to all tests that I have done... I have not seen anyone demonstrating a vmxnet3 NIC working in Nexenta 3.1.2 (modified or otherwise), and this issue has been brought up several times on these forums. Can you please provide a reference?

Matt,

I posted a few weeks ago that I show that on an ESXi5 U1 ESX host using the Nexentastor Enterprise Trial virtual appliance version 3.1.2 VM , vmxnet 3 is working and also supports an MTU of 9000. It appears to be already part of the Enterprise VM that nexenta provides for download. I also tried creating a VM from scratch and installing 3.1.2 and then installing vmware tools but I never could get vmxnet 3 to not error out with "macalloc() failed". E1000 worked fine.

My replies are at the bottom of this thread: http://nexentastor.org/boards/1/topics/6226

If there is another way for me to validate/demonstrate this for you, please let me know. I am running a series of nested ESXi host on a single physical host with Nexentastor providing NFS for VM storage on SSDs. The Nexentastor VM is on a standard vswitch alongside a VMKernel port and no uplinks. Both the VMKernel port and the Nexentastor storage adapter (vmxnet3) are set with an MTU of 9000. I tried E1000 in this same config and the vmxnet3 seemed to perform better.

If you want to verify this you could always download the Enterprise Trial VM appliance and give it a shot as well. HTH.

Doug Hagewood wrote:

Matt,

I posted a few weeks ago that I show that on an ESXi5 U1 ESX host using the Nexentastor Enterprise Trial virtual appliance version 3.1.2 VM , vmxnet 3 is working and also supports an MTU of 9000. It appears to be already part of the Enterprise VM that nexenta provides for download. I also tried creating a VM from scratch and installing 3.1.2 and then installing vmware tools but I never could get vmxnet 3 to not error out with "macalloc() failed". E1000 worked fine.

My replies are at the bottom of this thread: http://nexentastor.org/boards/1/topics/6226

If there is another way for me to validate/demonstrate this for you, please let me know. I am running a series of nested ESXi host on a single physical host with Nexentastor providing NFS for VM storage on SSDs. The Nexentastor VM is on a standard vswitch alongside a VMKernel port and no uplinks. Both the VMKernel port and the Nexentastor storage adapter (vmxnet3) are set with an MTU of 9000. I tried E1000 in this same config and the vmxnet3 seemed to perform better.

If you want to verify this you could always download the Enterprise Trial VM appliance and give it a shot as well. HTH.

Thanks Doug! I have not touched the Enterprise Trial in a long time, since i'm only currently using the Community Edition installed from ISO. I'm glad to hear that it works for one system, so hopefully it would not be too much work to get vmxnet3 working on the Community Edition for everyone. However, I feel my original point is still valid... VMXNet3 does not work out of the box (on Community Edition), which I am fairly sure the majority if Nexenta's non-production evaluation user base runs on.

Otherwise your test setup seems very similar to what I run in my home lab. I am glad to hear you can personally validate vmxnet3's superior performance over e1000.

I suppose it might be possible to compare the enterprise trial edition with the community edition to see why vmxnet3 run on one and not the other. but i'd prefer to not have to go through all that trouble (and I imagine Nexenta doesn't want people poking around under the covers that much either). If I get motivated and go through this process i will be sure to post my results.

There really should be no differences in enterprise vs community except for the plugins.

Linda Kateley wrote:

There really should be no differences in enterprise vs community except for the plugins.

I thought that as well. However my direct experience tells me vmxnet3 definitely does not work out of the box on Nexentastor Community Edition 3.1.2 when built from the ISO available here http://downloads.nexenta.com/cdn/NexentaStor-Community-3.1.2.iso.zip. I have also confirmed vmxnet3 still does not work after executing the setup appliance upgrade command on the console (resulting in the latest patch build).

If Doug says the vmxnet3 drivers work on the Enterprise Trial 3.1.2 VM image (i assume he downloaded it here: http://downloads.nexenta.com/cdn/NexentaStor-Enterprise-3.1.2-vmware.tar.bz2) I believe him. His screenshots seem genuine and I don't know why anyone would want to falsify a claim like that.

The simplest explanation is that some crafty Nexenta employee is performing some extra steps after installing Nexenta but before exporting/packaging the VMWare image that hasn't been documented. Linda, can you ask around your engineering staff to confirm this was the case, and ask them to share what was done to make vmxnet3 work?

I find this really interesting because vmxnet3 is supposed to be coming in 3.1.3. I have done alot of checking to figure out how they are seeing it now with EE.

Nexenta Support is adamant here is a code base difference between Community edition and Enterprise edition. I ran into this trouble shooting a NFS NIS authentication issue and a AD Join issue.