[Pvfs2-users] Pvfs performance over 10Ge

[Rob Ross]

Were netperf and iperf tests with TCP or UDP? Did you do any tuning when 
running those tools?

To answer your original question, no there's nothing in PVFS that is 
purposefully limiting you to 1Gbit/sec.

Thanks,

rob

Salmon, Rene wrote:
> Alll the pvfs servers are attached via a 4Gbit/sec fiber channel card to some RAID disks.
> 
> The disk arrays have xfs as the file system on them and a dd to the filesystem gives me about 350MBytes/sec per controller.
> 
> Each pvfs server is attached to 2 raid controllers so in theory I should be able to write at about 600 or more MBytes/sec on each pvfs server.
> 
> Also each pvfs server has an IB card and I ran the IB performance test earlier an saw speeds around 400-500MBytes/sec with pvfs2-cp and dd.
> 
> Thanks for the help 
> Rene
> 
> --------------------------
> Rene
> 
> 
> ----- Original Message -----
> From: Rob Ross <rross at mcs.anl.gov>
> To: Salmon, Rene
> Cc: pvfs2-users at beowulf-underground.org <pvfs2-users at beowulf-underground.org>
> Sent: Tue Oct 30 03:10:22 2007
> Subject: Re: [Pvfs2-users] Pvfs performance over 10Ge
> 
> How fast is the disk you're writing to? -- Rob
> 
> Salmon, Rene wrote:
>> Hi,
>>
>> Trying to benchmark pvfs running over 10ge.  I can run network
>> benchmarks like netperf and iperf and get numbers fairly close to
>> 10Gbits/sec  from any client to any of the pvfs servers.
>>
>> When I try to run some dd's or pvfs2-cp's on the pvfs file system I seem
>> to be reaching a 1Gbit/sec limit:
>>
>> hpca4000(salmr0)60:pvfs2-cp -t -b 4194304 /pvfs2-fs/foo /pvfs2-fs/junk3
>> Wrote 4194304000 bytes in 49.089332 seconds. 81.484099 MB/seconds
>>
>> hpca4000(salmr0)61:/usr/bin/time -p dd if=/dev/zero of=/pvfs2-fs/foo2
>> bs=4M count=1000
>> 1000+0 records in
>> 1000+0 records out
>> 4194304000 bytes (4.2 GB) copied, 50.4064 seconds, 83.2 MB/s
>>
>> Is there some hard coded limit with pvfs2 that assumes a 1Gbit/sec link
>> or am I missing something here?
>>
>> Thanks
>> Rene
>>
>> _______________________________________________
>> Pvfs2-users mailing list
>> Pvfs2-users at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>