[Pvfs2-users] re: [pvfs2-user] fail to use pvfs2-cp in 2.8.1

Rob Ross rross at mcs.anl.gov
Tue Jun 2 11:16:50 EDT 2009 

Really looks like a firewall problem to me. -- Rob

On Jun 2, 2009, at 9:19 AM, Phil Carns wrote:

> That is a little strange that you are getting a communication error on
> the client side (broken pipe) without the server also seeing a
> communication problem.  Is there anything unusual about your  
> network, or
> is it just a typical ethernet configuration?
>
> You may have to turn on some verbose logging for us to get a better  
> idea
> of what exactly is failing.  On the client side, you can set the
> PVFS2_DEBUGMASK environment variable to "verbose" and re-run your
> pvfs2-cp command to get client side logging output to stderr.
>
> On the server side you can enable verbose logging by changing
> "EventLogging none" to "EventLogging verbose" in the fs.conf file and
> restarting the server.  That will generate a lot of output, so you
> probably only want to leave it like that long enough to capture one
> pvfs2-cp attempt.
>
> -Phil
>
> Huang, Bin wrote:
>> It just says:
>>
>> [D 12/31 19:01] PVFS2 Server version 2.8.1 starting.
>>
>> Shouldn't it display something regarding my previous
>> pvfs2-ln operation? Hmm...
>>
>> My log is too simple to be a good log.
>>
>> Ben
>>
>> -----Original Message-----
>> From: Kevin Harms [mailto:harms at alcf.anl.gov]
>> Sent: 6/1/2009 () 16:14
>> To: Huang, Bin
>> Cc: Rob Ross; pvfs2-users at beowulf-underground.org
>> Subject: Re: [pvfs2-user] fail to use pvfs2-cp in 2.8.1
>>
>>
>>   Do you see an messages in the server log? By default it should / 
>> tmp/
>> pvfs2-server.log.
>>
>> kevin
>>
>> On Jun 1, 2009, at 2:58 PM, Huang, Bin wrote:
>>
>>> Hi, Rob & list,
>>>
>>> In my test, I have two nodes called n61 and n62.
>>> I ran my pvfs2 server on n62. When I was testing
>>> pvfs2-cp, I tried to run it from both n61 and n62.
>>> Both approaches gave me that pvfs2-cp failure.
>>>
>>> As I said, only pvfs2-cp has such problem. For
>>> pvfs2-ping and pvfs2-ls, they work well either
>>> locally or remotely.
>>>
>>> Interestingly, I can also do pvfs2-ln from the
>>> client. It seems that the client (my n61) is
>>> able to create a soft link in the server node (my n62).
>>>
>>> I am wondering if pvfs2-cp is depending on
>>> any specific system library which I might miss
>>> installing in my kernel.
>>>
>>> Briefly, here is what I am using:
>>> Linux version 2.6.26
>>> PowerPC405 (No FPU)
>>> 512MB DDR2
>>> GLibc 2.9
>>>
>>> (It looks pretty odd, I know. But it is part of
>>> our research)
>>>
>>> Any idea?
>>>
>>> Ben
>>>
>>>
>>> -----Original Message-----
>>> From: Rob Ross [mailto:rross at mcs.anl.gov]
>>> Sent: 5/29/2009 () 17:10
>>> To: Huang, Bin
>>> Cc: Kevin Harms; pvfs2-users at beowulf-underground.org
>>> Subject: Re: [Pvfs2-users] [pvfs2-user] fail to use pvfs2-cp in  
>>> 2.8.1
>>>
>>> Hi,
>>>
>>> Did you run this on a node other than n62?
>>>
>>> Rob
>>>
>>> On May 29, 2009, at 3:57 PM, Huang, Bin wrote:
>>>
>>>> Hi, kevin and list,
>>>>
>>>> When I look at my output from pvfs2-ping, there
>>>> is a false sense of security...
>>>>
>>>> Here is my output:
>>>>
>>>> root:~# /usr/local/bin/pvfs2-ping -m /mnt/pvfs2
>>>>
>>>> (1) Parsing tab file...
>>>>
>>>> (2) Initializing system interface...
>>>>
>>>> (3) Initializing each file system found in tab file: /etc/fstab...
>>>>
>>>>   PVFS2 servers: tcp://n62:3334
>>>>   Storage name: pvfs2-fs
>>>>   Local mount point: /mnt/pvfs2
>>>>   /mnt/pvfs2: Ok
>>>>
>>>> (4) Searching for /mnt/pvfs2 in pvfstab...
>>>>
>>>>   PVFS2 servers: tcp://n62:3334
>>>>   Storage name: pvfs2-fs
>>>>   Local mount point: /mnt/pvfs2
>>>>
>>>>   meta servers:
>>>>   tcp://n62:3334
>>>>
>>>>   data servers:
>>>>   tcp://n62:3334
>>>>
>>>> (5) Verifying that all servers are responding...
>>>>
>>>>   meta servers:
>>>>   tcp://n62:3334 Ok
>>>>
>>>>   data servers:
>>>>   tcp://n62:3334 Ok
>>>>
>>>> (6) Verifying that fsid 1186947128 is acceptable to all servers...
>>>>
>>>>   Ok; all servers understand fs_id 1186947128
>>>>
>>>> (7) Verifying that root handle is owned by one server...
>>>>
>>>>   Root handle: 1048576
>>>>     Ok; root handle is owned by exactly one server.
>>>>
>>>>
>>>>
>>>> Ben
>>>>
>>>> -----Original Message-----
>>>> From: Kevin Harms [mailto:harms at alcf.anl.gov]
>>>> Sent: Fri 5/29/2009 16:31
>>>> To: Huang, Bin
>>>> Cc: pvfs2-users at beowulf-underground.org
>>>> Subject: Re: [Pvfs2-users] [pvfs2-user] fail to use pvfs2-cp in
>>> 2.8.1
>>>>
>>>>
>>>>   What is the output from pvfs2-ping -m /mnt/pvfs2
>>>>
>>>> kevin
>>>>
>>>> On May 29, 2009, at 3:24 PM, Huang, Bin wrote:
>>>>
>>>>> Hi, list,
>>>>>
>>>>> I am currently testing my pvfs2-2.8.1 installation. I posted
>>>>> an error of failing to do pvfs2-ping on the list several
>>>>> weeks ago. It turned out that the issue came from lack of
>>>>> TCP wrapper in my kernel. Don't blame me for that because I
>>>>> was doing Linux from scratch and there are lots of ambiguous
>>>>> utilities to understand. :)
>>>>>
>>>>> Now I am facing another issue during my pvfs2 testing. When I
>>>>> was running pvfs2-cp, either when copying files to or from
>>>>> a PVFS2 file system, it failed to receive the package and the
>>>>> connection was refused.
>>>>>
>>>>> It is a little frustrating because this testing seems quite
>>>>> straightforward. Since I am doing Linux from scratch, I am
>>>>> not 100% sure my kernel will be compatible with pvfs2-2.8.1.
>>>>> I assume that people have tested pvfs2 with various mainstream
>>>>> distributions but not with a build-all-components-manually
>>>>> kernel before releasing it.
>>>>>
>>>>> I have no problem running pvfs2-ls, pvfs2-rm, pvfs2-mkdir,
>>>>> pvfs2-ln and pvfs2-fsck. So I am struggling to know whether
>>>>> this issue is from my kernel or from pvfs2 itself. I attached
>>>>> the error message hoping that someone can give me a guide.
>>>>>
>>>>> Here is my error message when using pvfs2-cp:
>>>>>
>>>>> root:~# /usr/local/bin/pvfs2-cp -t /usr/lib/libc.a /mnt/pvfs2/
>>>> testfile
>>>>> [  167.181678]   Signal: b
>>>>> [  167.181686]   Code:   30001
>>>>> [  167.181694]   Addr:   1a0
>>>>> [  167.188761]   Signal: b
>>>>> [  167.188769]   Code:   30001
>>>>> [  167.188777]   Addr:   1a0
>>>>> [E 03:13:52.731545] Receive immediately failed: Broken pipe
>>>>> [E 03:13:52.733795] Warning: msgpair failed to tcp://n62:3334,
>>> will
>>>>> retry: Broken pipe
>>>>> [E 03:13:52.734314] *** msgpairarray_completion_fn: msgpair to
>>>>> server tcp://n62:3334 failed: Broken pipe
>>>>> [E 03:13:52.734756] *** No retries requested.
>>>>> [E 03:13:52.743444] Receive immediately failed: Connection refused
>>>>> [E 03:13:52.744111] Warning: msgpair failed to tcp://n62:3334,
>>> will
>>>>> retry: Connection refused
>>>>> [E 03:13:52.744689] Warning: msgpair failed to tcp://n62:3334,
>>> will
>>>>> retry: Connection refused
>>>>> [E 03:13:52.745124] *** msgpairarray_completion_fn: msgpair to
>>>>> server tcp://n62:3334 failed: Connection refused
>>>>> [E 03:13:52.745494] *** No retries requested.
>>>>> [E 03:13:52.745898] *** msgpairarray_completion_fn: msgpair to
>>>>> server tcp://n62:3334 failed: Connection refused
>>>>> [E 03:13:52.746269] *** No retries requested.
>>>>> pvfs2-cp: src/client/sysint/sys-getattr.sm:309:
>>>>> getattr_acache_lookup: Assertion `object_ref.handle !=
>>>>> ((PVFS_handle)0)' failed.
>>>>> Aborted
>>>>>
>>>>>
>>>>>
>>>>> Benjamin Huang
>>>>> Reconfigurable Computing System lab
>>>>> UNC Charlotte
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Pvfs2-users mailing list
>>>>> Pvfs2-users at beowulf-underground.org
>>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Pvfs2-users mailing list
>>>> Pvfs2-users at beowulf-underground.org
>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>
>>>
>>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Pvfs2-users mailing list
>> Pvfs2-users at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users

More information about the Pvfs2-users mailing list