[Pvfs2-developers] pvfs2-cp profile

[Scott Atchley]

On Jan 11, 2007, at 3:28 PM, Scott Atchley wrote:

> On Jan 11, 2007, at 2:47 PM, Sam Lang wrote:
>
>>>> So this 11 msec delay could be caused by writing to disk, but if  
>>>> you don't have data sync enabled (which I don't think you do),  
>>>> then 11 msec for 1 MB is about 90MB/s to write data to buffer  
>>>> cache and return the write ack response.  We might be delayed  
>>>> here by having to wait for flow buffers being free'd up from one  
>>>> of the first 8 1MB messages.  Could you increase the  
>>>> FlowBuffersPerFlow option in the fs.conf to 16 or 20 and see  
>>>> what happens?
>>>>
>>>> -sam
>>>
>>> I will try this.
>>>
>>> I know that the default for FlowBuffersPerFlow is 8 per the  
>>> webpage, but I count 10 per flow in the log. I also find that  
>>> when using the -b flag, that it performs best with multiples of  
>>> 10 buffers.
>>
>> You're doing 10MB requests with 1MB flow buffers.  I think the 10  
>> flows per request are from that, so the number of flow buffers is  
>> still 8, it just has to reuse 2 of them.
>>
>> -b specifies the request size doesn't it?  Using multiples of 10  
>> for that value improves performance?  I really have no idea, but  
>> would have guessed that multiples of the flow buffer size (in your  
>> case 1MB) would have been better.
>>
>> -sam
>
> With FlowBuffersPerFlow set to 20, I am roughly the same gaps as  
> before (~11 msec between last send completion and the recv  
> completion and ~18 msec between transfers).
>
> Scott

On the server, I am seeing the exact same values. ~11 msec between  
the completion of the last recv before sending the ack and ~18 msec  
between sending the ack and the unexpected recv to start the next  
transfer.

Scott