[Pvfs2-users] iozone get different write bandwidth at write/rewrite

[neutron]

The "FlowBufferSizeBytes" by default is 262144.

I set it to 1M, and doesn't see a difference.

Thanks a lot Kyle!


-Neutron


On Fri, Apr 30, 2010 at 8:58 PM, Kyle Schochenmaier <kschoche at gmail.com> wrote:
> I believe each iozone process is writing its own file...
>
> Im curious to see if  pvfs2.8.2 addresses any of this, would it be
> possible to try it out ?
>
> As a general note with IB hardware - and it probably wont help with
> this specific problem - you may also want to set FlowBufferSizeBytes
> in your config file up to 1MB from 64k.
>
> Best,
>
> Kyle Schochenmaier
>
>
>
> On Fri, Apr 30, 2010 at 7:39 PM, neutron <neutronsharc at gmail.com> wrote:
>> Thanks Kyle!
>>
>> I disabled the "TroveSyncMeta" to "no", but it doesn't make a difference.
>> The poster you mentioned said the problem occurs with serial programs,
>> and parallel program wont' see such a hit.
>> what's  serial/parallel here mean?   I let 8 iozone processes run on 8
>> nodes and aggregate the bandwidth. Isn't it a parallel program?
>>
>> http://www.beowulf-underground.org/pipermail/pvfs2-users/2009-April/002770.html
>>
>>
>>
>> -Neutron
>>
>>
>> On Fri, Apr 30, 2010 at 8:17 PM, Kyle Schochenmaier <kschoche at gmail.com> wrote:
>>> Hi Neutron -
>>>
>>> Check out a very similar thread from about a year ago :
>>> http://www.beowulf-underground.org/pipermail/pvfs2-users/2009-April/002770.html
>>>
>>> Im not sure if we found out what was actually causing the performance
>>> differences..  but its probably a start to look into.
>>>
>>> Best,
>>> Kyle Schochenmaier
>>>
>>>
>>>
>>> On Fri, Apr 30, 2010 at 7:08 PM, neutron <neutronsharc at gmail.com> wrote:
>>>> Hi all,
>>>>
>>>> I have a pvfs2-2.8.1 with 4 DS.  "BMIModules" is set to  "bmi_ib".  I
>>>> use simple-tripe mode with stipe-size =1MB.
>>>>
>>>> 8 client nodes mount pvfs2, and I use iozone to measure the write
>>>> throughput, with each client node runs one iozone process.
>>>> Each process writes 1GB file with block size = 4MB.
>>>>
>>>> I got the following results:
>>>>
>>>>        Test running:
>>>>
>>>>        Children see throughput for  8 initial writers  =   98513.53 KB/sec
>>>>        Min throughput per process                      =   11406.64 KB/sec
>>>>        Max throughput per process                      =   12642.13 KB/sec
>>>>        Avg throughput per process                      =   12314.19 KB/sec
>>>>        Min xfer                                        =  946176.00 KB
>>>>        CPU Utilization: Wall time   86.492    CPU time    6.355    CPU
>>>> utilization   7.35 %
>>>>
>>>>
>>>>        Test running:
>>>>        Children see throughput for  8 rewriters        = 2499467.09 KB/sec
>>>>        Min throughput per process                      =  289479.62 KB/sec
>>>>        Max throughput per process                      =  321507.38 KB/sec
>>>>        Avg throughput per process                      =  312433.39 KB/sec
>>>>        Min xfer                                        =  946176.00 KB
>>>>        CPU utilization: Wall time    3.272    CPU time    5.990    CPU
>>>> utilization 183.08 %
>>>>
>>>>
>>>> We can see the "initial write"  gets very low throughput, while the
>>>> "rewrite" gives very high bandwidth.
>>>> My questions are:
>>>> 1. Why 8 client nodes  as a whole get low throughput?
>>>> 2. Why at "rewrite" phase, the write bandwidth boosts ?
>>>>
>>>>
>>>>
>>>> -Neutron
>>>> _______________________________________________
>>>> Pvfs2-users mailing list
>>>> Pvfs2-users at beowulf-underground.org
>>>> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>>>>
>>>
>>
>