[Pkg-ofed-devel] lenny - openmpi problems

[Yann JOBIC]

Guy Coates wrote:
> Yann JOBIC wrote:
>> Hello,
>>
>> I pluged 2 opteron SMP, with infiniband cards ( CA type: MT25418 ).
>> I've got a lenny distrib (5.0.2)
>> I used the deb http://pkg-ofed.alioth.debian.org/apt/ofed repository. 
>> I recompiled the modules for the kernel, just to be sur.
>> I directly connected 2 infiniband cards, with opensm running on both 
>> (one is master, the other slient)
>>
>> The ping test is fine.
>>
>> However, the bencharks included are not working :
>> Lidia:~# ib_rdma_lat Lilou
>> mlx4: There is a mismatch between the kernel and the userspace 
>> libraries: Kernel does not support XRC. Exiting.
>> 7312:pp_init_ctx: Couldn't get context for mlx4_0
>>
>
> Hi,
>
> That message means that your system is not picking up the new ofa 
> kernel modules you installed.
>
> Stupid question; did you reboot the machine/rmmod the existing 
> infiniband modules before running the tests?
>
>
> You can check which modules you are using  by running:
>
> modinfo mlx4_ib
>
> It should pick up the module in
>
>
> /lib/modules/<kernel-version>/updates/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 
>
>
>
> Can you send me the output of
>
> uname -r
>
> dpkg -l | grep ofa-kernel
>
> dpkg -s libibverbs1
>
> Cheers,
>
> Guy
>
Hi,

Thanks for the fast answer !

I rebooted the computer, yes.

I think i'm using the right module :
Lidia:~# modinfo mlx4_ib
filename:       
/lib/modules/2.6.26-2-amd64/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko
version:        1.0
license:        Dual BSD/GPL
description:    Mellanox ConnectX HCA InfiniBand driver
author:         Roland Dreier
srcversion:     F40D6BD2C4544C8E6E1CDB0
depends:        mlx4_core,ib_mad,ib_core
vermagic:       2.6.26-2-amd64 SMP mod_unload modversions

Lidia:~# uname -r
2.6.26-2-amd64

Lidia:~# dpkg -l | grep ofa-kernel
ii  ofa-kernel-source                   1.4.2-2                  Source 
for InfiniBand kernel drivers

Lidia:~# dpkg -s libibverbs1
Package: libibverbs1
Status: install ok installed
Priority: extra
Section: libs
Installed-Size: 88
Maintainer: Guy Coates <gmpc at sanger.ac.uk>
Architecture: amd64
Source: libibverbs
Version: 1.1.2-1+OFED
Depends: libc6 (>= 2.7-1), adduser
Description: A library for direct userspace use of RDMA (InfiniBand/iWARP)
 libibverbs is a library that allows userspace processes to use RDMA
 "verbs" as described in the InfiniBand Architecture Specification and
 the RDMA Protocol Verbs Specification.  iWARP ethernet NICs support
 RDMA over hardware-offloaded TCP/IP, while InfiniBand is a
 high-throughput, low-latency networking technology.  InfiniBand host
 channel adapters (HCAs) and iWARP NICs commonly support direct
 hardware access from userspace (kernel bypass), and libibverbs
 supports this when available.
 .
 For this library to be useful, a device-specific plug-in module
 should also be installed.
 .
 This package contains the shared library.
Homepage: http://www.openfabrics.org/

Cheers,

Yann