We have some ib built into the kernel, but most of it is not exposed in the gui. We have a requirement for it in an upcoming release. We are still very interested in use case.

Can i ask what you are using ib for?

lk

Looking at using it instead of 10 gigabit ... looks much cheaper for switches, while offering higher bandwidth.

Was wondering if anyone had this running in a production system.

Thanks,

Disclaimer: I am a Nexenta reseller.

I have two customers that are running Nexenta filers connected by IB. One has a DDR setup and the other has QDR. You have to go under the hood to make it work but once configured, it works famously.

I'm running nexenta with vmware over infiniband (ddr) srp. Waiting on mellanox to release drivers for esxi 5 so we can upgrade.

Hi,

I just want to step into this discussion and add us as a potential Nexenta user. About the use-cases asked by Linda, here are ours:

We've built up a cloud environment (currently focusing on IaaS at the beginning) with the following basic design:

• physical servers with Debian running VMs with no local storage
• dedicated physical servers for the storage backend (HA/clustering is a high priority for us)
• QDR Mellanox cards handling storage as well as network traffic (no separate storage backlinks - all in one mesh)

We tried out Nexenta and it first seemed to be very good. However, then we got a lot of problems lately (mainly around iSER) and reverted first to Solaris 10 and went forward to Solaris 11. However, we would really like to go with Open-Source solutions where we can get our hands on, as we have a lot of C/C++ developers, but none really into Solaris/IllumOS.

So, if someone could share some field-reports, problems that occurred and they got solved it would be nice to have a place for this. We also think that Infiniband is the way to go (infrastructure costs in total for our QDR 40 Gb/s setup is much less than with 10GbE).

I'll also try to collect all the details of the problems we had so far.

Also don't hesitate to ask for more details.

Thanks Ancoron, I think the iser is the piece that is missing and will need to be added to nexentastor. We don't currently support IB in nexentastor product, but are looking to in the future.

It might be in the other opensource distro from illumos. There is smart, openindiana and we will send out illumian in a few weeks. If you would like to test the ib in illumian let me know i will see if we can get you early access.

Hi Linda,

thanx for the quick response and yes, we would like to get our hands on new code. However, as we are currently just about to go into production most of our resources are already occupied, so it could take some time before I get results.

I'll also ask our staff about the details of the problems we've had with OpenIndiana, as I know that they've tested that as well.

Hi folks, I'm a new Nexenta evaluator, building up a server with four XenServer clients, would like to attach clients as SRP, but will fall back to IPoIB without issue. My switch doesn't have a subnet manager, and it would be nice if the Nexenta instance could provide that (would avoid having to juggle it in VMs).

Nexenta looks great, but this is one detail I really need to have solved. I'll give it a shot before asking questions, just wanted to chime in that there were others interested in this as well. Cheers!

I believe we got concensus to start working towards this in the product. It will be awhile but it is most likely on the way.

linda

That's great news, Linda! If you look at the maturity, flexibility and price/performance of Infiniband, it seems like a technology who's time has come.

Just another person running Infiniband without issues, windows servers storing SQL databases primarily.. Using SRP.

Nice to hear Steve, thanks for the report

For those running vmware, the infiniband driver for 5.0 has been delayed to may. So we running SRP still has a few weeks left on 4.1.