[Pvfs2-users] Problem with Mpich and PVFS2 when using InfiniBand
ligon at clemson.edu
Tue Mar 22 13:22:29 EST 2011
Can you give us the full command line when you configure PVFS? Seems that
you are doing this in two steps and must be done in one.
HPC Admin Staff
> I have installed PVFS (version 2.8.2) with InfiniBand support on a set of
> nodes running Debian (linux 2.6). The configuration line being the
> ./configure --with-openib=/usr --without-bmi-tcp --prefix=/usr
> The filesystem works fine with the kernel module, I'm able to write and
> to/from the filesystem.
> Then I compiled Mpich2 (1.2.1p1) with PVFS support using --enable-romio
> --with-pvfs2=/usr (among other options which doesn't matter).
> Running the IOR benchmark with POSIX, it works fine, but when using MPIIO
> got the following errors (this is from process 0, but all the processes
> printing the same)
> 0: libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
> 0: This will severely limit memory registrations.
> 0: [E 14:12:05.314874] Warning: openib_ib_initialize: ibv_open_device.
> 0: [E 14:12:05.314908] PVFS_isys_fs_add: Failed to initialize any
> appropriate BMI methods for addresses:
> 0: [E 14:12:05.314921] ib://parapide-7-ib0.rennes.grid5000.fr:3335
> 0: [E 14:12:05.314952] PVFS_isys_fs_add call: No such device
> 0: [E 14:12:05.314969] ERROR: could not initialize any file systems in
> 0: PVFS_util_init_defaults: No such device (error class: 0)
> Do you have any idea where it comes from? I'm unsuccefully trying to
> reinstall OFED-1.5, recompiling PVFS, Mpich, but I always get the same
> Note: previsouly everything worked well with Ethernet, I was able to have
> kernel module AND MPI-IO working.
> Thank you,
> Matthieu Dorier
> Matthieu Dorier
> ENS Cachan, Brittany (Computer Science dpt.)
> IRISA Rennes, Office E324
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
More information about the Pvfs2-users