[Pvfs2-developers] Re: [Pvfs2-users] PVFS2 on Infiniband

[Lee Whatley, Contractor]

Lee Whatley, Contractor wrote:
> Pete Wyckoff wrote:
>> I think it's best if I get around to doing the event-driven bmi_ib
>> rather than polling and see if that magically fixes it.  Playing
>> thread scheduling tricks will get us in trouble, as Nathan points
>> out.
> 
> Well, I'm hoping to upgrade this cluster from RHEL3 (2.4 kernel) to 
> RHEL4 (2.6 kernel) sometime in the next few months.  I have a feeling 
> alot of my problems will go away once that is done.

Hey Pete,

FYI I was finally cleared to upgrade my cluster to RHEL4 (2.6 kernel). 
Unfortunately this doesn't look like it fixed my problem.  Doing any 
operations on a pvfs2 filesystem over native infiniband (i.e. not tcp or 
IPoIB) are extermely slow.  Just a simple "ls" on a pvfs2 filesystem 
with a handful of files and directories takes 5-10 seconds and the 
pvfs2-server process takes up 98% of the CPU.

Because of the operational demands of the users on this cluster I can't 
  change the filesystems back from tcp to ib and get you some debug info 
right this moment.  I'm hoping I can set up some playspace where I can 
give you some more details later this week.

I'll keep you posted,
-Lee