[Pvfs2-developers] RE: PVFS2 seems tcp inclined ?

[Kyle Schochenmaier]

Looks like I forgot to CC pvfs2-dev a message or two ago, I'll put
this back on the list.


On Mon, Sep 1, 2008 at 6:30 PM, Kumar, Amit H. <AHKumar at odu.edu> wrote:
> Kyle:
> Sorry for the delayed response.
>>>> Also, to answer your other question, BMI_ib used to use tcp (not
>>>> bmi_tcp) to setup connections and do administrative stuff, but you
>>>> should not ever see 'file data'  go over your Ethernet connection.
>>>
>>> Okay. In that case connection setup should work without Ethernet interface being up.
>>> Which I don't see happening with the options I used to configure PVFS2.
>>I'm not sure if  bmi_ib uses IB to do its connection management, there
>>was talk of porting it to use openIB's connection manager but I think
>>I was the only one who was serious about doing it, and thus it never
>>got done to my knowledge.
>
> I disabled ethernet interface just to test that, and I found it doesn't use bmi_ib for connection management.
>
I didnt think this was implemented yet. (as it was my potential job to do it)
>>>
>>> As far as I notice when I use regular copy command, not pvfs2-cp: I see the traffic going
>>> through the Ethernet interface. IMPORTANT THING TO NOTE:  I have disabled Ethernet
>>> interfaces completely on my PVFS2 IO & Metadata servers. Mysterious and interesting...!!!
>>>
>>Just to be clear, when you use pvfs2-cp, you do not get traffic over
>>the ethernet device?
>
> I do get traffic over the ethernet device. Though I noticed one problem with pvfs2-cp as far as the installation I have.
> when I do # pvfs2-cp source local_filesystem_file  path_to_pvfs2_FS/.
> The pvfs2-cp command fails with a Segmentation Fault.
> Where As when I chage directory to PVFS2 filesystem and then copy as shown below, it succeeds:
> [/scratch/pvfs2 #] pvfs2-cp path_to_sourcefilesystem_file ./.
>
> Any idea ?? Does this sound like a bug ???
Can you provide a backtrace?
Also I believe there are a couple fixes to IB that have occured since
2.7.1 that dealt with possible segfaults.
Can you please try the latest cvs and see if you get the same segfault issues?
Lets start with that if its possible, just to make sure we're not
stepping on our own toes here.


Kyle

>
>>Is it possible that you somehow have a bmi_tcp client mounted somewhere?
>
> No I have double checked it.
>
>>What version of pvfs2 & kernel are you using, and do you have
>>infiniband setup properly?
>
> PVFS2: version 2.7.1
> Kernel: 2.6.18-53.1.14.el5
>



-- 
Kyle Schochenmaier