[Pvfs2-developers] libpvfs2 usage

Rob Ross rross at mcs.anl.gov
Thu Oct 19 15:28:11 EDT 2006


Sure! I think that the small values really only make sense for systems 
where TCP is being used with small kernel buffer sizes.

We're working on a tuning document describing starting points for many 
of these parameters on various types of systems.

Rob

Scott Atchley wrote:
> On Oct 19, 2006, at 1:12 PM, Sam Lang wrote:
> 
>> So the 256K chunks you see are from that default flow buffer size.  We 
>> only post bmi operations in flow buffer sized chunks, so at the BMI 
>> method layer (IB in this case) on the server, you see those memory 
>> registrations.  On the client the buffer passed in through the system 
>> interfaces is used, so I would bet that the memory registrations don't 
>> get chunked into flow buffer sized regions like they do on the server.
>>
>> To change the flow buffer size, you can add this to your fs.conf:
>>
>> FlowBufferSizeBytes 1446864
>>
>> It needs to be added within the <Filesystem> context.
>>
>> -sam
> 
> This will make a difference on MX as well. If you want to get the most 
> performance from a 10 Gb/s link, you want bulk transfers of 512 KB or 
> (preferably) more.
> 
> Scott


More information about the Pvfs2-developers mailing list