[PVFS2-developers] latest mpi-io-test numbers on jazz

[Robert Latham]

On Thu, Mar 31, 2005 at 07:45:40PM +0200, Phil Carns wrote:
> There definitely needs to be some improvement here.  I think the most 
> important write graph is the "nosync" version, though.  According to the 
> rough calculations in the earlier email, that graph shows us getting 23 
> MB/s per server, which is close to 75% of peak.  Still not as good as we 
> could be, though.

How smart is the request scheduler these days?  

> I don't think that the runs with TroveSync set to yes really tell us 
> that much.  Those are forcing trove to sync after every individual 
> underlying operation (which can be no larger than 256K each by default) 
> whether the application wants it to or not.  That's adding a good bit of 
> superfluous work to a benchmark that is already syncing the traditional 
> way by just calling a sync() function once at the end of a write.  It is 
> also probably causing blocks to flush to disk in an unusual order given 
> the interleaving of data from many clients, therefore causing extra disk 
> latency.

Wow, I didn't realize we were syncing after every 256K bytes!  

> PVFS2 ships with the default configuration files setting TroveSync to 
> yes, though, so I guess that's really showing PVFS2's normal mode of 
> operation.  IMHO, I tend to think that turning TroveSync off is a better 
> default choice.  Maybe at least until we have a server controlled cache 
> that prevents us from hitting trove unless we really want to hit 
> physical disk.  Applications still have the choice of deciding when/if 
> to explicitly sync regardless of the TroveSync option.

I'm trying to learn how frequently berkely db will flush data if
clients never call db->sync.  Even though we've worked out a lot of
sofware reasons for server crashes, we still see hardware blowups.
Just curious how exposed people are to data loss if sync is off.

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B