[Pvfs2-developers] acache timeout

[Dean Hildebrand]

Thanks for the info Phil.  While I'm running an I/O performance 
experiment, I never want the attributes to expire, so I'm using the -a 
option to set it to a really high value. 

But for the average pvfs2 setup, what is a useful value?  With 5msec, 
the cache was continually expiring, and my data servers were contacting 
the metadata server multiple times when writing a single file (which 
takes on the order of seconds, if not minutes for large files), just to 
retrieve the same information over and over.

I've seen single client file write throughput (with < 10 of data 
servers) achieve roughly 40-60MB/s (with gigabit ethernet).  So taking a 
1G file around 20s to write to disk.  Of course there is no typical 
setup or typical file size, but maybe a better default acache timeout 
value is in the 10s of seconds?
Dean

Phil Carns wrote:
> Dean Hildebrand wrote:
>> Hi All,
>>
>> I noticed my acache was never being used, and so I realized there is 
>> a discrepancy between
>> #define ACACHE_DEFAULT_TIMEOUT_MSECS 
>> <ident?v=pvfs2-1.5.1-server;i=ACACHE_DEFAULT_TIMEOUT_MSECS> 5000
>> and the default timeout passed in to pvfs2-client (which is '-a 5').
>>
>> Is the intention when using the kernel module to reset the cache 
>> timeout to 5msecs?  I'd prefer to stick with 5 seconds.
>
> I have no idea why things are set up this way, but it has been like 
> that for a while.  I helped rework the acache code at some point and 
> left the same settings in place so that my patch didn't change semantics.
>
> The pvfs2-client.c file defines its own defaults that it passes along 
> to pvfs2-client-core unless you tell it otherwise:
>
> #define DEFAULT_ACACHE_TIMEOUT_STR "5"
>
> The easiest way to change it would be to edit your startup script for 
> pvfs2-client to pass in --acache-timeout 5000 or -a 5000.  You can 
> also change the parameter dynamically using 
> /proc/sys/pvfs2/acache/timeout-msecs once pvfs2-client is running.
>
> Bart Taylor posted some patches to the ncache that give you similar 
> functionality for the name lookup cache (ie, command line and proc 
> control of the timeout parameters) among other things, but I don't 
> think it has been integrated in the tree yet.
>
> -Phil
>
>
>

-- 
Dean Hildebrand
Ph.D. Candidate
University of Michigan