[Pvfs2-users] Problem with Mpich and PVFS2 when using InfiniBand

[Kevin Harms]

  you need to modify your 'limit'.

  ulimit -l unlimited

  then ulimit -l to check it.

  if you can change it, you might need to tweak something as root in the limits.conf

kevin

On Mar 22, 2011, at 12:05 PM, Matthieu Dorier wrote:

> Hello,
> 
> I have installed PVFS (version 2.8.2) with InfiniBand support on a set of nodes running Debian (linux 2.6). The configuration line being the following:
> ./configure --with-openib=/usr --without-bmi-tcp --prefix=/usr
> The filesystem works fine with the kernel module, I'm able to write and read to/from the filesystem.
> Then I compiled Mpich2 (1.2.1p1) with PVFS support using --enable-romio --with-pvfs2=/usr (among other options which doesn't matter).
> Running the IOR benchmark with POSIX, it works fine, but when using MPIIO I got the following errors (this is from process 0, but all the processes are printing the same)
> 
> 0: libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
> 0:     This will severely limit memory registrations.
> 0: [E 14:12:05.314874] Warning: openib_ib_initialize: ibv_open_device.
> 0: [E 14:12:05.314908] PVFS_isys_fs_add: Failed to initialize any appropriate BMI methods for addresses:
> 0: [E 14:12:05.314921]     ib://parapide-7-ib0.rennes.grid5000.fr:3335
> 0: [E 14:12:05.314952] PVFS_isys_fs_add call: No such device
> 0: [E 14:12:05.314969] ERROR: could not initialize any file systems in /etc/mtab.
> 0: PVFS_util_init_defaults: No such device (error class: 0)
> 
> Do you have any idea where it comes from? I'm unsuccefully trying to reinstall OFED-1.5, recompiling PVFS, Mpich, but I always get the same thing.
> Note: previsouly everything worked well with Ethernet, I was able to have kernel module AND MPI-IO working.
> 
> Thank you,
> 
> Regards
> 
> Matthieu Dorier
> 
> 
> 
> -- 
> Matthieu Dorier
> ENS Cachan, Brittany (Computer Science dpt.)
> IRISA Rennes, Office E324
> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2909 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20110322/df165f0b/smime.bin