[Pvfs2-users] PVFS2 over infiniband: pvfs2-client-core uses up 100% CPU

[Vikrant Kumar]

Hello Pete

Thanks for the info. I think we'll have to stick to one protocol only in 
the face of these issues.

The pvfs2-IB cluster is up finally and there is some speedup as compared 
to a pvfs2-TCP setup on a different cluster. But Im seeing one strange 
thing on the IB setup: the pvfs2-client-core is using up 100% CPU even 
in the absence of file-system activity. I saw on pvfs2 FAQs one entry 
related to pvfs2-server CPU usage, but none related to 
pvfs2-client-core. Have other people run into a similar problem? Note 
that on the pvfs2-TCP cluster, which is similarly configured except for 
not having Infiniband, pvfs2-client-core is quite well behaved.

The pvfs2 is setup as follows:
IO servers: node{0-3}
metadata servers : node0
protocol : IB
clients: node{0-3}
note that Im running the clients on the same machines as the servers, 
each a dual proc.
OS : Linux SLES9 x86_64
kernel: 2.6.5-7.244-smp

Let me know if anyone has faced this problem or has a solution.

Thanks
Vikrant


Pete Wyckoff wrote:
> vsk at fluent.co.in wrote on Fri, 25 Aug 2006 19:35 +0530:
>   
>>    I was able to mount the file-system after compiling the client with IB
>>    support. Thanks : )
>>    Now another question that comes up is: If I have one cluster with pvfs2-IB
>>    and another with pvfs2-TCP and I'm using mpich2, the question is can I
>>    compile mpich2 so that applications(compiled against this mpich2) can work
>>    on both pvfs2-IB and pvfs2-TCP? What happens when vendors differ for IB?
>>    What happens when we add pvfs2-GM clusters to the mix?
>>    Im  essentially looking at one mpich2 build to be able to handle these
>>    scenarios.
>>     
>
> (Your emails are html and very hard to read.  Please use plain text
> only in the future.)
>
> You will have mpich2 problems as well, if you expect users to use IB
> or TCP or GM for message passing in their clusters.  This is a hard
> problem for all software vendors.  Shipping source is your best bet.
>
> Failing that, shipping binaries that expect certain mpich2 and pvfs2
> shared libraries may work.  We can't help you with much of this, but
> if you identify a specific problem in pvfs2 that keeps it from
> working for your multi-device setup, let us know and we'll see if
> it's fixable.
>
> 		-- Pete
>
>