[Pvfs2-users] Problem with Mpich and PVFS2 when using InfiniBand
harms at alcf.anl.gov
Tue Mar 22 16:30:34 EST 2011
you need to modify your 'limit'.
ulimit -l unlimited
then ulimit -l to check it.
if you can change it, you might need to tweak something as root in the limits.conf
On Mar 22, 2011, at 12:05 PM, Matthieu Dorier wrote:
> I have installed PVFS (version 2.8.2) with InfiniBand support on a set of nodes running Debian (linux 2.6). The configuration line being the following:
> ./configure --with-openib=/usr --without-bmi-tcp --prefix=/usr
> The filesystem works fine with the kernel module, I'm able to write and read to/from the filesystem.
> Then I compiled Mpich2 (1.2.1p1) with PVFS support using --enable-romio --with-pvfs2=/usr (among other options which doesn't matter).
> Running the IOR benchmark with POSIX, it works fine, but when using MPIIO I got the following errors (this is from process 0, but all the processes are printing the same)
> 0: libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
> 0: This will severely limit memory registrations.
> 0: [E 14:12:05.314874] Warning: openib_ib_initialize: ibv_open_device.
> 0: [E 14:12:05.314908] PVFS_isys_fs_add: Failed to initialize any appropriate BMI methods for addresses:
> 0: [E 14:12:05.314921] ib://parapide-7-ib0.rennes.grid5000.fr:3335
> 0: [E 14:12:05.314952] PVFS_isys_fs_add call: No such device
> 0: [E 14:12:05.314969] ERROR: could not initialize any file systems in /etc/mtab.
> 0: PVFS_util_init_defaults: No such device (error class: 0)
> Do you have any idea where it comes from? I'm unsuccefully trying to reinstall OFED-1.5, recompiling PVFS, Mpich, but I always get the same thing.
> Note: previsouly everything worked well with Ethernet, I was able to have kernel module AND MPI-IO working.
> Thank you,
> Matthieu Dorier
> Matthieu Dorier
> ENS Cachan, Brittany (Computer Science dpt.)
> IRISA Rennes, Office E324
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 2909 bytes
Desc: not available
Url : http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20110322/df165f0b/smime.bin
More information about the Pvfs2-users