[Pkg-ofed-devel] OFED on Debian squeeze

[Ira Weiny]

On Fri, 20 Jan 2012 04:21:22 -0800
Matthieu Labbé <matthieu.labbe at gmail.com> wrote:

> Hello,
> 
> I need to get InfiniBand working on a Debian squeeze cluster.

My recommendation would be to abandon OFED for all the user space components and build them from source.

tarballs here: http://www.openfabrics.org/downloads/
git trees here: http://git.openfabrics.org/git

As for the kernel, I don't know if there was a backport in OFED for that distro.  However, it is likely to have a backport which is close you could start from.  There is a risk though that OFED contains some things which are not accepted upstream and are likely to change as you move forward.  I don't know if OFED 1.4 had these changes or not.  The main issue was the addition of "XRC" support to verbs.  The Open Fabrics community is working through getting this upstream, and may even be there now.

> 
> The manual at http://pkg-ofed.alioth.debian.org/howto
> and associated  package at http://pkg-ofed.alioth.debian.org/apt/ofed
> do not work for me (fails on dependencies, I can provide log if needed).
> Google is not very helpful either...
> 
> Questions:
> 1) What is the status of OFED on Debian squeeze? Is there a repository available ?
> 2) Does anybody have experience to share installing from the source from openfabrics.org<http://openfabrics.org> ? (They provide instruction for CentOS.)
> 3) If installing from source, which version should I choose ?
> 4) What is the relation of the "infiniband-diags" package (http://packages.debian.org/fr/squeeze/infiniband-diags) with OFED ? Does its version number, 1.4.4-20090314-1, refer to the OFED version ?
> 

The infiniband-diags package version "loosely" corresponds to the OFED version number.  The problem with OpenSM and the diags is the ABI for libibumad and libibmad were very much in flux in the OFED 1.4 timeframe.  libibumad is much more stable now but libibmad is likely to continue changing as the specification has a lot of errata updates for MAD packets.  Because of this debian and ubuntu seem to be stuck on an extremely old version of those libraries.

I have recently built the latest management "stack" on ubuntu 11.04.  (See instructions for building from git trees here: http://www.openfabrics.org/downloads/management/README)  The latest versions are: http://www.openfabrics.org/downloads/management/latest.txt

With those packages you should get most functionality on OFED 1.4 and full functionality on OFED 1.5.

If you are successful and have knowledge of building deb packages I would appreciate patches to support building libibmad and infiniband-diags on debian systems.  I have wanted to do this for a long time but have not had the time.  I am pretty sure I could help get similar patches accepted in OpenSM and libibumad as well.

Hope this helps,
Ira


> Any advice would be appreciated.
> 
> Thanks,
> Matt.


-- 
Ira Weiny
Member of Technical Staff
Lawrence Livermore National Lab
925-423-8008
weiny2 at llnl.gov