[PVFS2-users] PVFS2 performance

[dschwenk at iastate.edu]

Just curious what kind of interconnect you are using?

David Schwenker


> Hi all,
> 
> I just installed PVFS2 in a test 4 nodes cluster (head node metadata server, the
> four nodes IO nodes). Also, following the instructions on section 5 of the quick
> start guide, I installed the kernel interface in the four machines, and run the
> pvfs2-client. It all seems OK, I can see /mnt/pvfs2 from all machines, and I can
> copy, move, etc. files around with the regular commands: ls, cp, etc.
> 
> Now, just copying (with cp) a big file to /mnt/pvfs2 or to another place through
> NFS gives much better performance to NFS, but I understand that's probably
> normal, so I try to run the mpi-io-test at
> ftp://ftp.parl.clemson.edu/pub/pvfs2/tests/
> 
> I have mpich-1.2.6, the code compiles with no problems and then I test.
> 
> First, to a NFS mounted directory:
> 
> [beo at nodo0 ~]$ mpirun -np 4 mpi-io-test -i 10 -f /net/nodo0/scratch/foo.out -p
/etc/pvfs2tab
> # Using mpi-io calls.
> nr_procs = 4, nr_iter = 10, blk_sz = 16777216
> # total_size = 671088640
> # Write: min_t = 5.339317, max_t = 66.259956, mean_t = 50.615525, var_t =
911.269773
> # Read:  min_t = 1.053671, max_t = 48.641617, mean_t = 36.042611, var_t =
544.747562
> Write bandwidth = 10.128118 Mbytes/sec
> Read bandwidth = 13.796594 Mbytes/sec
> 
> And then to /mnt/pvfs2
> 
> [beo at nodo0 ~]$ mpirun -np 4 mpi-io-test -i 10 -f /mnt/pvfs2/foo.out -p
/etc/pvfs2tab
> # Using mpi-io calls.
> nr_procs = 4, nr_iter = 10, blk_sz = 16777216
> # total_size = 671088640
> # Write: min_t = 43.714029, max_t = 46.666999, mean_t = 45.611818, var_t =
1.691665
> # Read:  min_t = 44.097879, max_t = 47.578564, mean_t = 46.287230, var_t =
2.755476
> Write bandwidth = 14.380368 Mbytes/sec
> Read bandwidth = 14.104853 Mbytes/sec
> [beo at nodo0 ~]$
> 
> 
> As you can see, the gain is not great. Is this the way to run it? Shouldn't I
> get a better gain? Do I have to install the kernel interface for these tests?
> 
> Any light into measuring performance with PVFS2 and what I should expect? 
> 
> Thanks,
> Angel de Vicente
> -- 
> ----------------------------------
> http://www.iac.es/galeria/angelv/
> 
> PostDoc Software Support
> Instituto de Astrofisica de Canarias
> 
> _______________________________________________
> PVFS2-users mailing list
> PVFS2-users at beowulf-underground.org
> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
>