[Pvfs2-users] Problem with Mpich and PVFS2 when using InfiniBand

Chris Poultney crispy at cs.nyu.edu
Tue Mar 22 14:17:29 EST 2011


Is make kmod_install also required?

-crispy


On 03/22/2011 03:04 PM, Matthieu Dorier wrote:
> If you mean the command line for pvfs2-genconfig, here it is:
>
> /usr/bin/pvfs2-genconfig --quiet --protocol ib --ioservers "parapide-7-ib0"
> --metaservers "parapide-7-ib0" --storage /tmp/pvfs2-storage-space --logfile
> /tmp/pvfs2-log --fsname pvfs2 pvfs2-fs.conf
>
> I'm basically following the "Quick start guide".
>
> If you mean the configuration before compiling, here it is:
>
> ./configure --prefix=/usr --with-kernel=/lib/modules/2.6.26-2-amd64/build
> --with-openib=/usr --without-bmi-tcp
> (then make, make kmod, make install)
>
> Matthieu
>
> 2011/3/22 Becky Ligon<ligon at clemson.edu>
>
>> Can you give us the full command line when you configure PVFS?  Seems that
>> you are doing this in two steps and must be done in one.
>>
>> Becky
>> --
>> Becky Ligon
>> HPC Admin Staff
>> PVFS Developer
>> Clemson University
>> 864-650-4065
>>
>>> Hello,
>>>
>>> I have installed PVFS (version 2.8.2) with InfiniBand support on a set of
>>> nodes running Debian (linux 2.6). The configuration line being the
>>> following:
>>> ./configure --with-openib=/usr --without-bmi-tcp --prefix=/usr
>>> The filesystem works fine with the kernel module, I'm able to write and
>>> read
>>> to/from the filesystem.
>>> Then I compiled Mpich2 (1.2.1p1) with PVFS support using --enable-romio
>>> --with-pvfs2=/usr (among other options which doesn't matter).
>>> Running the IOR benchmark with POSIX, it works fine, but when using MPIIO
>>> I
>>> got the following errors (this is from process 0, but all the processes
>>> are
>>> printing the same)
>>>
>>> 0: libibverbs: Warning: RLIMIT_MEMLOCK is 32768 bytes.
>>> 0:     This will severely limit memory registrations.
>>> 0: [E 14:12:05.314874] Warning: openib_ib_initialize: ibv_open_device.
>>> 0: [E 14:12:05.314908] PVFS_isys_fs_add: Failed to initialize any
>>> appropriate BMI methods for addresses:
>>> 0: [E 14:12:05.314921]     ib://parapide-7-ib0.rennes.grid5000.fr:3335
>>> 0: [E 14:12:05.314952] PVFS_isys_fs_add call: No such device
>>> 0: [E 14:12:05.314969] ERROR: could not initialize any file systems in
>>> /etc/mtab.
>>> 0: PVFS_util_init_defaults: No such device (error class: 0)
>>>
>>> Do you have any idea where it comes from? I'm unsuccefully trying to
>>> reinstall OFED-1.5, recompiling PVFS, Mpich, but I always get the same
>>> thing.
>>> Note: previsouly everything worked well with Ethernet, I was able to have
>>> kernel module AND MPI-IO working.
>>>
>>> Thank you,
>>>
>>> Regards
>>>
>>> Matthieu Dorier
>>>
>>>
>>>
>>> --
>>> Matthieu Dorier
>>> ENS Cachan, Brittany (Computer Science dpt.)
>>> IRISA Rennes, Office E324
>>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
>>> _______________________________________________
>>> Pvfs2-users mailing list
>>> Pvfs2-users at beowulf-underground.org
>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>
>>
>>
>>
>
>
>
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users


More information about the Pvfs2-users mailing list