[Pvfs2-users] performance problem

[Phil Carns]

Could you compare the following three ways to copy a file onto PVFS 
(without ftp or samba for the moment) to see if it is specific to "cp"?

a) time cp /tmp/file.dat /mnt/pvfs/file1.dat

b) time dd if=/tmp/file.dat of=/mnt/pvfs/file2.dat bs=8M

c) time pvfs2-cp /tmp/file.dat /mnt/pvfs/file3.dat

If there is a big difference between a) and b), it may just be that your 
version of cp (ie, coreutils package) is using a 4KB block size 
regardless of what the file system reports.  If there is a big 
difference between b) and c), it may indicate a kernel module problem. 
If all three are slow then there must be a more fundamental issue.

-Phil


Emmanuel Florac wrote:
> Hi, I've made a pvfs2 cluster to agregate storage from 2 machines. The
> machines run Linux Debian etch with 2.6.22.19 linux. All is well, both
> machines are identical with 3Ware raid controllers, and write disk
> thruput is about 550MB/s.
> Each machine has 4 GigEth ports. I've dedicated 2 of them to link the
> machines together (bonding, 802.3ad) for the pvfs traffic, the 2 others
> are connected to the outside world. The purpose is to provide a rough
> ftp/samba storage thru the VFS layer, and it works fine.
> 
> However the performance is abysmal : the network thruput between the
> nodes is 8MB/s while copying! the total cluster write performance
> (cp from local disk to pvfs storage) is 17 MB/s.... What can be going so
> wrong? Any hint would be appreciated.
>