[Pvfs2-users] Pvfs performance over 10Ge

[Sam Lang]

Hi Rene,

With both ib and tcp methods enabled, both methods are getting polled  
in succession.  This is a current limitation in PVFS, and we've seen  
pretty much the same behavior that you've described, so disabling the  
IB stuff should help fix your bandwidth limitations.

-sam

On Oct 30, 2007, at 1:56 PM, Rene Salmon wrote:

> Hi Scott,
>
> I have pvfs compiled and configured to use both bmi_ib and bmi_tcp  
> (will
> probaly try MX later):
>
>         BMIModules bmi_ib,bmi_tcp
>
>         Alias hpcxe001a ib://hpcxe001:3335,tcp://hpcxe001:3334
>         Alias hpcxe001b ib://hpcxe001:3337,tcp://hpcxe001:3336
>         Alias hpcxe003a ib://hpcxe003:3335,tcp://hpcxe003:3334
>         Alias hpcxe003b ib://hpcxe003:3337,tcp://hpcxe003:3336
>
>
> I guess maybe its trying to use tcp over IB or something.  I will rip
> out the IB stuff so there is no chance of that and try again.
>
> Just looked in cvs and did not see the bmi_pingpong test do you mind
> sending me a copy?
>
> Thanks
> Rene
>
>
> On Tue, 2007-10-30 at 11:08 +0000, Scott Atchley wrote:
>> On Oct 29, 2007, at 11:32 PM, Rob Ross wrote:
>>
>>> To answer your original question, no there's nothing in PVFS that
>>> is purposefully limiting you to 1Gbit/sec.
>>
>> Hi Rene,
>>
>> When I used bmi_pingpong to test the network IO performance of PVFS2
>> (ignoring the file system parts) on Myri-10G Ethernet, bmi_tcp
>> achieved 625-650 MB/s using a single client and single server:
>>
>> http://www.myri.com/scs/performance/MX-10G/PVFS2-MX/
>>
>> Performance of pvfs2-cp will be lower, but it should be above 1 Gb/s.
>>
>> Scott
>>
>>
> _______________________________________________
> Pvfs2-users mailing list
> Pvfs2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>