[Pvfs2-developers] pvfs2-cp performance with single client and single server

[Scott Atchley]

Hi Rob,

Doing a non-disk test with Lustre, I hit 1170 MB/s using 3-4 threads  
(single client to/from single server).

Using a single thread with Lustre, I get 675 MB/s.

Using the same machines as below, my Lustre results are at:

https://mail.clusterfs.com/wikis/lustre/MX-10G

Scott

On Jan 8, 2007, at 2:08 PM, Rob Ross wrote:

> Nice Scott! What's the realistic BW limit for this hardware?
>
> Rob
>
> Scott Atchley wrote:
>> On Jan 5, 2007, at 11:47 AM, Scott Atchley wrote:
>>> I modified perf.c to write 10 times a buffer of 128 MBs. I get:
>>>
>>> % mpiexec -n 1  ./perf -fname pvfs2:/mnt/pvfs2/x1
>>> Access size per process = 134217728 bytes, ntimes = 10
>>> Write bandwidth without file sync = 827.824860 Mbytes/sec
>>> Read bandwidth without prior file sync = 870.979531 Mbytes/sec
>>> Write bandwidth including file sync = 827.166730 Mbytes/sec
>>> Read bandwidth after file sync = 865.624838 Mbytes/sec
>>>
>>> using a single client. The client used about 25% CPU.
>>>
>>> Scott
>> The above was with 1 MB buffers. With 16 MB buffers:
>> % mpiexec -n 1  ./perf -fname pvfs2:/mnt/pvfs2/x1
>> Access size per process = 268435456 bytes, ntimes = 10
>> Write bandwidth without file sync = 822.867597 Mbytes/sec
>> Read bandwidth without prior file sync = 844.154572 Mbytes/sec
>> Write bandwidth including file sync = 823.773517 Mbytes/sec
>> Read bandwidth after file sync = 844.393557 Mbytes/sec
>> With 4 MB buffers:
>> % mpiexec -n 1  ./perf -fname pvfs2:/mnt/pvfs2/x1
>> Access size per process = 268435456 bytes, ntimes = 10
>> Write bandwidth without file sync = 847.215804 Mbytes/sec
>> Read bandwidth without prior file sync = 862.068630 Mbytes/sec
>> Write bandwidth including file sync = 847.861378 Mbytes/sec
>> Read bandwidth after file sync = 862.307479 Mbytes/sec
>> The client CPU usage was 25% for both values. I do not see any  
>> benefit for larger than 1 MB buffers.
>> Scott
>> _______________________________________________
>> Pvfs2-developers mailing list
>> Pvfs2-developers at beowulf-underground.org
>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers