[PVFS2-users] Low performance of PVFS2 on our small cluster

[Robert Latham]

On Wed, Oct 27, 2004 at 12:21:49AM +0200, Silvestre Zabala wrote:
> The summary result is 
> "b_eff_io =   25.604 MB/s on 10 processes with 500 MByte/PE"
> (The full result and graphics are attached.)
> 
> This is about the same we get with our simple test programs - and about
> the same we would expect from using b_eff_io on a NFS mount.
> (Unfortunately, we can't get it to run with NFS at the moment).
> 
> We hope someone can help us tracking the problem and/or supply us with
> b_eff_io results on similar systems.

So you've got a 5 node cluster running PVFS2.  Are you running
b_eff_io on these same 5 nodes, from a separate 5 node cluster (two
processes per node) or from a separate 10 node cluster?

> b_eff_io.c, Revision 2.1 from Dec. 12, 2001
> 
> MEMORY_PER_PROCESSOR =  500 MBytes  [1MBytes = 1024*1024 bytes, 1MB = 1e6 bytes]
> Maximum chunk size   =    3.906 MBytes
> 
> -N  10 T=60, MT=5000 MBytes, -D WITHOUT_SHARED, -noshared, -rewrite
> PATH=pvfs2://p, PREFIX=b_eff_io
>        system name : Linux
>        hostname    : master1
>        OS release  : 2.4.27-1-686-smp
>        OS version  : #1 SMP Sun Sep 12 05:10:57 CEST 2004
>        machine     : i686
> 
> Date of measurement: Tue Oct 26 22:57:04 2004

Here's my summary running on our Jazz cluster, 15 nodes (5 io nodes,
10 clients), connected via myrinet.  It won't be a fair comparison,
but it's the closest I could do quickly :>

b_eff_io.c, Revision 2.1 from Dec. 12, 2001

MEMORY_PER_PROCESSOR = 1024 MBytes  [1MBytes = 1024*1024 bytes, 1MB = 1e6 bytes]
Maximum chunk size   =    8.000 MBytes

-N  10 T=60, MT=10240 MBytes, -D WITHOUT_SHARED, -noshared, -rewrite
PATH=pvfs2:/pvfs2-auto, PREFIX=PVFS2_10pe_0060sec_noshared
       system name : Linux
       hostname    : j2
       OS release  : 2.4.26
       OS version  : #1 SMP Wed Apr 21 09:09:56 CDT 2004
       machine     : i686

...
 b_eff_io := maximum over all bw_partition =   67.448 MB/s

 b_eff_io =   67.448 MB/s on 10 processes with 1024 MByte/PE, scheduled time=1.0 Min, on Linux j2 2.4.26 #1 SMP Wed Apr 21 09:09:56 CDT 2004 i686, NOT VALID (see above)

==rob

-- 
Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B