[Pvfs2-users] PVFS2 over inifiniband(pathscales infinipath)

[rishi pathak]

You are right, OpenIB supports infinipath.But the problem is that we have to
use pathscale's MPI libraries and they only works with pathscale's
driver.IfI go for OpenIB, then I would have to use
MVAPICH.I will centainly try out OpenIB on some of the nodes and will get
back to you as soon as possible with the results
Regards--
Rishi Pathak

On 1/5/07, Kyle Schochenmaier <kschoche at scl.ameslab.gov> wrote:
>
> rishi pathak wrote:
> > Hi Kyle,
> >      Thanks for replying.
> >            I am not using OpenIB, instead I am using libraries
> > provided by pathscale.
> >
> >
> We only currently have ports to OpenIB userverbs and Mellanox's VAPI
> interfaces, I cant speak for the rest of the developers, but I doubt I
> will be writing an additional port to the pathscale libraries.  However,
> if I recall correctly, OpenIB has a pathscale library, actually I know
> it does, as I tried to get some pathscale htx nics working yesterday and
> couldnt heh.
> If you'd like to try the openib port out, that would be very helpful to
> the development team in getting feedback on new architectures that we
> have yet to test.
>
> Good luck
>
>     +=Kyle
>
> > On 1/3/07, *Kyle Schochenmaier* <kschoche at scl.ameslab.gov
> > <mailto:kschoche at scl.ameslab.gov>> wrote:
> >
> >     rishi pathak wrote:
> >     >
> >     > Hi All,
> >     >    I am trying to install pvfs2 2.6.0 over infiniband.I am using
> >     > pathscales infinipath adapter.
> >     >    make and make install works fine and also the storage space
> gets
> >     > created.I have configured it to use ib(infiniband.)
> >     >    But when I try to run the servers normally they exit
> >     abnormally.The
> >     > log says that ib is unsupported protocol.
> >     >    Is pathscale's infinipath support not enabled in pvfs2, or
> >     should I
> >     > go with TCP and use layered ethernet driver for inifinipath.
> >     >    Below is my log:
> >     >
> >     > [D 21:03:46.121370] PVFS2 Server version 2.6.0 starting.
> >     > [E 01/03 21:03] src/io/bmi/bmi.c line 1840: Error: no method
> >     available
> >     > for bmi_ib.
> >     > [E 01/03 21:03]         [bt] /usr/local/pvfs2/sbin/pvfs2-server
> >     [0x41e890]
> >     > [E 01/03 21:03]         [bt]
> >     > /usr/local/pvfs2/sbin/pvfs2-server(BMI_initialize+0x13e)
> [0x41ed5e]
> >     > [E 01/03 21:03]         [bt] /usr/local/pvfs2/sbin/pvfs2-server
> >     [0x40df8d]
> >     > [E 01/03 21:03]         [bt]
> >     > /usr/local/pvfs2/sbin/pvfs2-server(main+0xaa1) [0x40f821]
> >     > [E 01/03 21:03]         [bt]
> >     > /lib64/tls/libc.so.6(__libc_start_main+0xdb) [0x332881c3fb]
> >     > [E 01/03 21:03]         [bt]
> >     > /usr/local/pvfs2/sbin/pvfs2-server(aio_cancel64+0x16a) [0x40d62a]
> >     > [E 01/03 21:03] Error: BMI_initialize: Protocol not available
> >     > [E 01/03 21:03] Error: Could not initialize server interfaces;
> >     aborting.
> >     > [E 01/03 21:03] Error: Could not initialize server; aborting
> >     >
> >     >
> >
> ------------------------------------------------------------------------
> >     >
> >     >
> >     If you're using OpenIB API I believe Infinipath is supported through
> >     user verbs, though I have yet to test this out.  I've also seen this
> >     message when not all of the kernel modules are loaded.  Make sure
> you
> >     have ib_uverbs.ko installed.  Hopefully this will clear it up.  If
> >     not,
> >     it looks like a config problem.
> >
> >     Kyle
> >     > _______________________________________________
> >     > Pvfs2-users mailing list
> >     > Pvfs2-users at beowulf-underground.org
> >     <mailto:Pvfs2-users at beowulf-underground.org>
> >     > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
> >     >
> >     >
> >     >
> >
> >
> >     --
> >     Kyle Schochenmaier
> >     kschoche at scl.ameslab.gov <mailto:kschoche at scl.ameslab.gov>
> >     Research Assistant, Dr. Brett Bode
> >     AmesLab - US Dept.Energy
> >     Scalable Computing Laboratory
> >
> >
> > !DSPAM:459c9a6d293411804284693!
>
>
> --
> Kyle Schochenmaier
> kschoche at scl.ameslab.gov
> Research Assistant, Dr. Brett Bode
> AmesLab - US Dept.Energy
> Scalable Computing Laboratory
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.beowulf-underground.org/pipermail/pvfs2-users/attachments/20070105/c3734bf0/attachment.htm