[PVFS2-users] Low performance of PVFS2 on our small cluster

Bradley W Settlemyer bradles at parl.clemson.edu
Tue Oct 26 19:56:22 EDT 2004


I got roughly 4MB/s in b_eff_io with MPICH(romio) + NFS, but perhaps you
are using a different MPI implementation and machine configuration (your
disks are faster than ours).

Others can probably provide better insight, but here is how I run
b_eff_io on our cluster.  I would start a *single* pvfs2 server on each
of your 5 IO nodes using all the memory for that node (1 gig in this
case).  I would then allocate 10 compute nodes with which to run
b_eff_io -- I would not bother running multiple processes per compute
node, as your FastEthernet interconnect is likely saturated.  So thats
15 nodes total (5 I/O Nodes and 10 b_eff_io nodes).  I'm not sure you'll
see a great deal of improvement (again I suspect FastEthernet is your
limiting factor), but you may see some if you were using a different


On Tue, 2004-10-26 at 18:21, Silvestre Zabala wrote:
> Hello all,
> we have been working with PVFS2 and while writing programs that access
> the FS through MPI-IO we've seen a rather low performance of PVFS2.
> Basically we have have seen a performance level similar to NFS.
> Now, we are pretty sure that this can't be the intention of the authors,
> and that we may have a configuration problem here. 
> Here is a quick description of our test cluster. It is built out of five
> Dual-Xeons at 2GHz with 1G memory and interconnected with Fast Ethernet.
> We are running the latest MPICH2 and PVFS2. The storage are is located
> on the local HDDs which, according to bonnie++, are able to read or
> write about 45 MB/s. So it's a pretty generic COTS-cluster.
> As we have seen pretty erratic and low performance results with our own
> simple tests, we ran the b_eff_io benchmark.
> The summary result is 
> "b_eff_io =   25.604 MB/s on 10 processes with 500 MByte/PE"
> (The full result and graphics are attached.)
> This is about the same we get with our simple test programs - and about
> the same we would expect from using b_eff_io on a NFS mount.
> (Unfortunately, we can't get it to run with NFS at the moment).
> We hope someone can help us tracking the problem and/or supply us with
> b_eff_io results on similar systems.
> Best regards,
>   Silvestre

More information about the PVFS2-users mailing list