[PVFS2-developers] latest mpi-io-test numbers on jazz

[Walter B. Ligon III]

--------

Can I use these charts in the talk at IPDPS Monday.

Walt

> A little bit ago I queued up a bunch of mpi-io-test jobs on jazz over
> the myrinet network and they finally finished.  
> 
> http://www.mcs.anl.gov/~robl/pvfs2/jazz-20050308/
> 
> [The -eb versions of the plots in that directory show the high, low,
> and average over multiple runs (4 or 5 i think, depending on how many
> jobs finished before Jazz recently had an "unscheduled maintenance" :> ]
> 
> As always for these mpi-io-test runs, these read numbers are with warm
> caches.  Given that favorable condition, we peaked out at about 8.5
> GiB/sec (128 servers and 100 clients).   
> 
> The big knee at 8 servers and 100 clients shouldn't be too alarming.
> That's the point where the clients working set exceeds the servers'
> memory (and caches).
> 
> Write numbers look rather unimpressive as usual.  The disks on Jazz
> compute nodes are nothing special.  The recent system software upgrade
> means we can compare the performance impact of TroveSyncMeta and
> TroveSyncData.  jazz-gm-write.png is with those config file options
> set to 'yes'.  jazz-gm-write-nosync is with them set to 'no'.  The
> mpi-io-test program syncs after every MPI_File_write, so we are
> still pushing data to disk, just not as often.   Rough calculations
> give us 18 MiB/sec per server with TroveSync{Meta,Data} set to 'yes', 
> but 23 MiB/sec per serfer with those options set to 'no'.   The
> TroveSync options did not appear to have any impact on read
> performance (nor would one expect to see such an impact).
> 
> The biggest change between this run and the previous run in September
> is that Jazz was recently upgraded to Redhat Enterprise Linux 3.
> pvfs2-server now gets to take advantage of working AIO callbacks in
> glibc.  Just as Neill asserted, aio callbacks make a significant
> difference in performance. 
> 
> For comparison, here are the old plots:
> http://www-unix.mcs.anl.gov/~robl/pvfs2/jazz_gm.read-20040928.png
> http://www-unix.mcs.anl.gov/~robl/pvfs2/jazz_gm.write-20040928.png
> 
> I guess I should write this up in a little more coherent manner but
> I'm somewhat pressed for time ... let me know if you would like
> anything explained in further detail.  
> 
> ==rob
> 
> -- 
> Rob Latham
> Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
> Argonne National Labs, IL USA                B29D F333 664A 4280 315B
> _______________________________________________
> PVFS2-developers mailing list
> PVFS2-developers at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
> 

-- 
Dr. Walter B. Ligon III
Associate Professor
ECE Department
Clemson University