[Pvfs2-developers] pvfs2-cp profile
atchley at myri.com
Thu Jan 11 15:46:27 EST 2007
On Jan 11, 2007, at 3:28 PM, Scott Atchley wrote:
> On Jan 11, 2007, at 2:47 PM, Sam Lang wrote:
>>>> So this 11 msec delay could be caused by writing to disk, but if
>>>> you don't have data sync enabled (which I don't think you do),
>>>> then 11 msec for 1 MB is about 90MB/s to write data to buffer
>>>> cache and return the write ack response. We might be delayed
>>>> here by having to wait for flow buffers being free'd up from one
>>>> of the first 8 1MB messages. Could you increase the
>>>> FlowBuffersPerFlow option in the fs.conf to 16 or 20 and see
>>>> what happens?
>>> I will try this.
>>> I know that the default for FlowBuffersPerFlow is 8 per the
>>> webpage, but I count 10 per flow in the log. I also find that
>>> when using the -b flag, that it performs best with multiples of
>>> 10 buffers.
>> You're doing 10MB requests with 1MB flow buffers. I think the 10
>> flows per request are from that, so the number of flow buffers is
>> still 8, it just has to reuse 2 of them.
>> -b specifies the request size doesn't it? Using multiples of 10
>> for that value improves performance? I really have no idea, but
>> would have guessed that multiples of the flow buffer size (in your
>> case 1MB) would have been better.
> With FlowBuffersPerFlow set to 20, I am roughly the same gaps as
> before (~11 msec between last send completion and the recv
> completion and ~18 msec between transfers).
On the server, I am seeing the exact same values. ~11 msec between
the completion of the last recv before sending the ack and ~18 msec
between sending the ack and the unexpected recv to start the next
More information about the Pvfs2-developers