[Pvfs2-developers] RE: PVFS2 seems tcp inclined ?

[Kumar, Amit H.]

Kyle:

>
> Amit -
> The config file you sent looks like it should work for IB without tcp?
> Which config options are you setting for the other cases?

All of my IO servers and Metadata server use the same config file, that I posted in my reply.
I am not sure what you mean by "which config options am I setting for other cases"?
If you mean while running the configure script, then I am not using any other config options
 apart from what I posted earlier.

>
> Also, to answer your other question, BMI_ib used to use tcp (not
> bmi_tcp) to setup connections and do administrative stuff, but you
> should not ever see 'file data'  go over your Ethernet connection.

Okay. In that case connection setup should work without Ethernet interface being up.
Which I don't see happening with the options I used to configure PVFS2.

As far as I notice when I use regular copy command, not pvfs2-cp: I see the traffic going
through the Ethernet interface. IMPORTANT THING TO NOTE:  I have disabled Ethernet
interfaces completely on my PVFS2 IO & Metadata servers. Mysterious and interesting...!!!


Thank you,
Amit

>
> Kyle
>
>
>
> On Thu, Aug 28, 2008 at 1:14 PM, Kumar, Amit H. <AHKumar at odu.edu>
> wrote:
> > My config config options:
> >
> > ./configure --prefix=/opt/pvfs2 --enable-karma --with-
> kernel=/usr/src/linux-2.6 --with-openib=/usr --with-openib-
> includes=/usr/include --with-openib-libs=/usr/lib64 CFLAGS=-g --enable-
> nptl-workaround=yes --without-bmi-tcp
> >
> > Thanks!!!
> > Amit
> >
> > -----Original Message-----
> > From: Kyle Schochenmaier [mailto:kschoche at gmail.com]
> > Sent: Thursday, August 28, 2008 1:55 PM
> > To: Kumar, Amit H.
> > Cc: pvfs2-developers at beowulf-underground.org
> > Subject: Re: [Pvfs2-developers] RE: PVFS2 seems tcp inclined ?
> >
> > Amit -
> > Can you send a copy of your .conf file so I can take a closer look at
> > what order you have things setup.
> > I'm interested in the BMI_Modules section and the alias' sections for
> > each of your above configurations.
> > Also, did you configure/build with `--without-bmi-tcp'?  (as
> suggested
> > for IB users)
> >
> > ~Kyle
> >
> > On Thu, Aug 28, 2008 at 12:15 PM, Kumar, Amit H. <AHKumar at odu.edu>
> wrote:
> >> Hello Again ...
> >>
> >>
> >>
> >> Can anyone please help me understand this ??
> >>
> >>
> >>
> >> Thank you,
> >>
> >> Amit
> >>
> >>
> >>
> >> From: pvfs2-developers-bounces at beowulf-underground.org
> >> [mailto:pvfs2-developers-bounces at beowulf-underground.org] On Behalf
> Of
> >> Kumar, Amit H.
> >> Sent: Tuesday, August 26, 2008 2:07 PM
> >> To: 'pvfs2-developers at beowulf-underground.org'
> >> Subject: [Pvfs2-developers] PVFS2 seems tcp inclined ?
> >>
> >>
> >>
> >> Hi All,
> >>
> >>
> >>
> >> On compute nodes when I have both Ethernet and IB interface up data
> traffic
> >> gets routed through the Ethernet interface as opposed to IB, though
> pvfs2 is
> >> built with IB, and tcp_bmi disabled.
> >>
> >> Also to note, when I disable Ethernet interface, it routes the
> traffic
> >> through IB interface.
> >>
> >>
> >>
> >> Is this how it should work ??? Any thoughts please.
> >>
> >>
> >>
> >> Thank you,
> >>
> >> Amit
> >>
> >>
> >>
> >> _______________________________________________
> >> Pvfs2-developers mailing list
> >> Pvfs2-developers at beowulf-underground.org
> >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
> >>
> >>
> >
> >
> >
> > --
> > Kyle Schochenmaier
> >
>
>
>
> --
> Kyle Schochenmaier
> _______________________________________________
> Pvfs2-developers mailing list
> Pvfs2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers